错误在轨道上运行scrape命令红宝石

Question

我正尝试在新计算机上重新设置我的应用程序并运行刮擦来构建数据库。当我运行我的第一把耙刮刀时：刮，这是我得到的错误。我不知道为什么我得到这个错误任何帮助将使我的一天..干杯！

Art West@ARTWESTIV ~/desktop/duckduckjeep-master 
$ rake scraper:scrape --trace 
** Invoke scraper:scrape (first_time) 
** Invoke environment (first_time) 
** Execute environment 
** Execute scraper:scrape 
rake aborted! 
NoMethodError: undefined method `value' for nil:NilClass 
c:/Users/Art West/desktop/duckduckjeep-master/lib/tasks/scraper.rake:17:in `block (3 levels) in <top (required)>' 
c:/Users/Art West/desktop/duckduckjeep-master/lib/tasks/scraper.rake:12:in `loop' 
c:/Users/Art West/desktop/duckduckjeep-master/lib/tasks/scraper.rake:12:in `block (2 levels) in <top (required)>' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:240:in `call' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:240:in `block in execute' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `each' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:235:in `execute' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:172:in `invoke_with_call_chain' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/task.rb:165:in `invoke' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:150:in `invoke_task' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block (2 levels) in top_level' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `each' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:106:in `block in top_level' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:115:in `run_with_threads' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:100:in `top_level' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:78:in `block in run' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run' 
c:/RailsInstaller/Ruby2.1.0/lib/ruby/gems/2.1.0/gems/rake-10.4.2/bin/rake:33:in `<top (required)>' 
c:/RailsInstaller/Ruby2.1.0/bin/rake:23:in `load' 
c:/RailsInstaller/Ruby2.1.0/bin/rake:23:in `<main>' 
Tasks: TOP => scraper:scrape 

这里是我的scraper.rake

namespace :scraper do 
    desc "Fetch Craigslist posts from 3Taps" 
    task scrape: :environment do 
    require 'open-uri' 
    require 'json' 
    # Set API token and URL 
    auth_token = "b077632d17da8857e2fa92c053115e43" 
    polling_url = "http://polling.3taps.com/poll" 

    # Grab data until up-to-date 
    loop do 

    # Specify request parameters 
    params = { 
     auth_token: auth_token, 
     anchor: Anchor.first.value, 
     source:"CRAIG", 
     category_group: "VVVV", 
     category: "VAUT", 
     'location.country' => "USA", 
     retvals: "location,external_url,heading,body,timestamp,price,images,annotations" 

    } 

    # Prepare API request 
    uri = URI.parse(polling_url) 
    uri.query = URI.encode_www_form(params) 

    # Submit request 
    result = JSON.parse(open(uri).read) 

    # Display results to screen 
    #puts result["postings"].first["annotations"]["year"] 

    Anchor.first.update(value: result["anchor"]) 
    puts Anchor.first.value 
    break if result["postings"].empty? 

    # #store results in Database 
    result["postings"].each do |posting| 

     #ADD HARD FILTER (IN PROGRESS....) 
     if posting["annotations"]["make"] == "Jeep" 

     #create new post 
     @post= Post.new 
     @post.heading = posting["heading"] 
     @post.body = posting["body"] 
     @post.price = posting["price"] 
     @post.neighborhood = posting["location"]["locality"] 
     @post.external_url = posting["external_url"] 
     @post.timestamp = posting["timestamp"] 
     @post.year = posting ["annotations"]["year"] if posting ["annotations"]["year"].present? 
     @post.make = posting ["annotations"]["make"] if posting ["annotations"]["make"].present? 
     @post.model = posting ["annotations"]["model"] if posting ["annotations"]["model"].present? 
     @post.title_status = posting ["annotations"]["title_status"] if posting ["annotations"]["title_status"].present? 
     @post.transmission = posting ["annotations"]["transmission"] if posting ["annotations"]["transmission"].present? 
     @post.mileage = posting ["annotations"]["mileage"] if posting ["annotations"]["mileage"].present? 
     @post.source_account = posting ["annotations"]["source_account"] if posting ["annotations"]["source_account"].present? 
     @post.phone = posting ["annotations"]["phone"] if posting ["annotations"]["phone"].present? 
     @post.lat = posting["location"]["lat"] 
     @post.lng = posting["location"]["long"] 
     @post.zipcode = posting["location"]["zipcode"] 
     #Save Post 
     @post.save 

     # Loop over images and save to Image database 
     posting["images"].each do |image| 
      @image = Image.new 
      @image.url = image["full"] 
      @image.post_id = @post.id 
      @image.save 
     end 
     end 

    end 
    end 
    end 




desc "Destroy All Posting Data" 
task destroy_all_posts: :environment do 
    Post.destroy_all 
end 

desc "Save neighborhood codes in a reference table" 
task scrape_neighborhoods: :environment do 
    require 'open-uri' 
    require 'json' 

     # Set API token and URL 
     auth_token = "b077632d17da8857e2fa92c053115e43" 
     location_url = "http://reference.3taps.com/locations" 

    # Specify request parameters 
    params = { 
     auth_token: auth_token, 
     level: "locality", 
     country: "USA" 
    } 


# Prepare API request 
uri = URI.parse(location_url) 
uri.query = URI.encode_www_form(params) 

    # Submit request 
    result = JSON.parse(open(uri).read) 

    # Display results to screen 
    # puts JSON.pretty_generate result 

    # Store results in database 
    result["locations"].each do |location| 
     @location = Location.new 
     @location.code = location["code"] 
     @location.name = location["short_name"] 
     @location.save 
    end 
    end 




    desc "Discard old data" 
    task discard_old_data: :environment do 
    Post.all.each do |post| 
     if post.created_at < 72.hours.ago 
     post.destoy 
     end 
    end 
    end 
end 

Answer 1

你的错误消息说这一切：

NoMethodError: undefined method `value' for nil:NilClass 

您试图在某些时候调用value上nil。而且，这可能是发生在这里：

# Specify request parameters 
params = { 
    auth_token: auth_token, 
    anchor: Anchor.first.value, # this is the line that's creating problem 
    source:"CRAIG", 
    category_group: "VVVV", 
    category: "VAUT", 
    'location.country' => "USA", 
    retvals: "location,external_url,heading,body,timestamp,price,images,annotations" 

} 

当调用此：anchor: Anchor.first.value的第一次，你Anchor.first是nil，所以你试图拨打：nil.value而这也正是你的rake任务失败，并指定错误信息。

请确保您已填充数据库表（anchors），以便在拨打Anchor.first时不会得到nil。

另一种方法来避免这个问题是使用try：

anchor: Anchor.first.try(:value) 

这样，你的rake任务不会失败，即使Anchor.first回报nil。