0
问题: 我有几个sidekiq线程和一个函数,只能从任何线程调用一次。同步一个方法不同的Sidekiq线程并等待
原因: 我们正在查询AdWords API以获取一些数据。当涉及到费率限制时,它们的限制性很强。只有一个线程可能会调用该函数来一次获取数据。
现在一些代码:
# Public: Get estimates for a set of keywords. If there is an error, retry
# several times. If not successful, raise an error
#
# keywords: The keyword objects to get estimates for.
# save: Boolean to indicate whether the keyword objects should be saved to
# the database
#
def repeatedly_try_get_estimates(keywords: [], save: true, sleep_delay: 150)
return keywords if keywords.empty?
func = -> { get_estimates(keywords, !save) }
retry_operation(function: func, max_tries: 15, sleep_delay: sleep_delay)
end
- 正如你所看到的,现在我有一个巨大的
sleep_delay
解决该问题。 - 代码调用
retry_operation
函数,get_estimates
函数作为参数。然后它将重试get_estimates
函数几次,直到出现API 异常。
的retry_function
:
# Private: Retry a function X times and wait X seconds. If it does not work X times,
# raise an error. If successful return the functions results.
#
# - max_tries: The maximum tries to repeat the function
# - sleep_delay: The seconds to wait between each iteration.
# - function: The lambda function to call each iteration
#
def retry_operation(max_tries: 5, sleep_delay: 30, function: nil, current_try: 0, result: nil)
# Can't call, no function
if function.nil?
return
end
# Abort, tried too frequently.
if current_try > max_tries
raise "Failed function too often"
end
# Check if there is an exception
exception = true
begin
result = function.call
exception = false
rescue => e
Rails.logger.info "Received error when repeatedly calling function #{e.message.to_s}"
end
if exception
sleep sleep_delay if sleep_delay > 0
retry_operation(max_tries: max_tries, sleep_delay: sleep_delay, function: function, current_try: current_try + 1)
else
result
end
end
的get_estimates_function
是在这里:https://gist.github.com/a14868d939ef0e34ef9f。这太长了,以防万一。
我想我需要做到以下几点:
- 调整代码在
repeatedly_try_get_estimates
功能。 - 在课堂上使用互斥锁。
- 如果正在使用互斥锁,请抢救异常。
- 只有当互斥是免费的,运行
rety_operation
,否则睡眠一段时间
感谢您的帮助:)
是你只用一个sidekiq进程和多个线程,还是有多个sidekiq进程在运行? –
这是一个单独的sidekiq进程,具有多个线程。 – Hendrik