用Elixir刮一个网站的问题

Question

我正在尝试使用我的应用程序来获得一个简单的猎犬测试，我想出了它与硒的错误。这是代码：

在mix.exs：

defmodule Scraper.Mixfile do 
 
    use Mix.Project 
 

 
    def project do 
 
    [app: :scraper, 
 
    version: "0.0.1", 
 
    elixir: "~> 1.0", 
 
    build_embedded: Mix.env == :prod, 
 
    start_permanent: Mix.env == :prod, 
 
    deps: deps] 
 
    end 
 

 
    # Configuration for the OTP application 
 
    # 
 
    # Type `mix help compile.app` for more information 
 
    def application do 
 
    [applications: [:logger, :httpoison, :hound]] 
 
    end 
 

 
    # Dependencies can be Hex packages: 
 
    # 
 
    # {:mydep, "~> 0.3.0"} 
 
    # 
 
    # Or git/path repositories: 
 
    # 
 
    # {:mydep, git: "https://github.com/elixir-lang/mydep.git", tag: "0.1.0"} 
 
    # 
 
    # Type `mix help deps` for more examples and options 
 
    defp deps do 
 
    [ 
 
     {:httpoison, "~> 0.7"}, 
 
     {:floki, "~> 0.7"}, 
 
     {:hound, "~> 0.7"} 
 
    ] 
 
    end 
 
end

在LIB/scraper.ex

defmodule Example do 
 
    use Hound.Helpers 
 

 
    def run do 
 
    Hound.start_session 
 
    IO.inspect "Iniciando" 
 
    navigate_to "http://akash.im" 
 
    IO.inspect page_title() 
 

 
    Hound.end_session 
 
    end 
 
end

在配置/ config.exs

# This file is responsible for configuring your application 
 
# and its dependencies with the aid of the Mix.Config module. 
 
use Mix.Config 
 

 
# This configuration is loaded before any dependency and is restricted 
 
# to this project. If another project depends on this project, this 
 
# file won't be loaded nor affect the parent project. For this reason, 
 
# if you want to provide default values for your application for third- 
 
# party users, it should be done in your mix.exs file. 
 

 
# Sample configuration: 
 
# 
 
#  config :logger, :console, 
 
#  level: :info, 
 
#  format: "$date $time [$level] $metadata$message\n", 
 
#  metadata: [:user_id] 
 

 
# It is also possible to import configuration files, relative to this 
 
# directory. For example, you can emulate configuration per environment 
 
# by uncommenting the line below and defining dev.exs, test.exs and such. 
 
# Configuration from the imported file will override the ones defined 
 
# here (which is why it is important to import them last). 
 
# 
 
#  import_config "#{Mix.env}.exs" 
 
# Define how long the application will wait between failed attempts (in miliseconds) 
 
config :hound, retry_time: 100000 
 
# Start with selenium driver (default) 
 
config :hound, driver: "selenium"

启动webdriver的服务器

java -jar selenium-server-standalone-2.45.0.jar

运行的应用程序：

/scraper$ iex -S mix 
 
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] 
 

 
Interactive Elixir (1.0.5) - press Ctrl+C to exit (type h() ENTER for help) 
 
iex(1)> Example.run 
 
** (exit) exited in: :gen_server.call(Hound.SessionServer, {:find_or_create_session, #PID<0.148.0>}, 60000) 
 
    ** (EXIT) an exception was raised: 
 
     ** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}} 
 
      (hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4 
 
      (hound) lib/hound/session_server.ex:22: Hound.SessionServer.handle_call/3 
 
      (stdlib) gen_server.erl:629: :gen_server.try_handle_call/4 
 
      (stdlib) gen_server.erl:661: :gen_server.handle_msg/5 
 
      (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3 
 

 
11:26:13.971 [error] GenServer Hound.SessionServer terminating 
 
Last message: {:find_or_create_session, #PID<0.148.0>} 
 
State: #HashDict<[]> 
 
** (exit) an exception was raised: 
 
    ** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}} 
 
     (hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4 
 
     (hound) lib/hound/session_server.ex:22: Hound.SessionServer.handle_call/3 
 
     (stdlib) gen_server.erl:629: :gen_server.try_handle_call/4 
 
     (stdlib) gen_server.erl:661: :gen_server.handle_msg/5 
 
     (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3 
 
    (stdlib) gen_server.erl:212: :gen_server.call/3 
 
    (scraper) lib/scraper.ex:37: Example.run/0 
 
iex(1)>

Answer 1

请求在这种情况下超时，这可以从线

** (MatchError) no match of right hand side value: {:error, %HTTPoison.Error{id: nil, reason: :timeout}} 

如果你看一下堆栈跟踪中可以看出，它表明错误是在

(hound) lib/hound/request_utils.ex:43: Hound.RequestUtils.send_req/4 

如果你打开猎犬源，上线的lib/hound/request_utils.ex 43你看

case type do 
    :get -> 
    {:ok, resp} = HTTPoison.get(url, headers, @http_options) 
    :post -> 
    {:ok, resp} = HTTPoison.post(url, body, headers, @http_options) 
    :delete -> 
    {:ok, resp} = HTTPoison.delete(url, headers, @http_options) 
end 

这种鳕鱼e预计会有回应，否则会崩溃。你的情况有一个超时错误，导致崩溃。

请检查运行测试时网站是否可用并重试。