when timeouts aren't timeouts 2
Last Friday, a good chunk of our team spent a while debugging a problem afflicting some sites on one of our shared hosting machines.
After a bunch of red herrings, some TCP dumps determined that the sites in question were all hung up on numerous DNS requests. That led us to the call to a web service we use on the sites in question. As it turns out, the IP geocode service being called, HostIP.info, was completely down. (It’s free; you get what you pay for.)
We assumed this couldn’t be the case, since those calls are wrapped in a Ruby Timeout calls with a maximum of 2 seconds each (Timeout.timeout(2)). Well, apparently, Ruby Timeout just doesn’t work sometimes.
Big props to Andy for sleuthing that down. And bigger props to Philippe Hanrigou for writing that very detailed article where he says, specifically:
“In particular, initiating network connections and/or a broken or slow DNS server will typically block the whole Ruby process while the call completes.”
Bingo! We’re going to try Philippe’s library on these projects and others in the future when we need guaranteed timeouts.
Comments
-
Philippe Hanrigou about 4 hours later:Hi Colin, glad you found the article useful. This is exactly the kind of problem that got us started on SystemTimer ;-) Let me know about your experience with the gem...
-
cheapRoc 17 days later:@Philippe, it would be great if the gem's RDoc or Yard would state that is has the same exception notices as Timeout. I was a bit confused right off the bat until I started using it and found that out. Other than that the gem works excellent!
