Add option to turn off waiting for delay when following meta redirects and option to follow meta redirects < a given # of seconds

[bmatthew]

It'd be really nice to have the option to turn off waiting for meta refresh delay for sites that have extremely long meta refreshes. A lot of news homepages do this, for example http://www.cnn.com/.

With the options currently available in Mechanize, I can either ignore all meta refreshes or try to follow them all with potentially a long wait (thread goes to sleep for meta refresh delay). For CNN, if I turn follow meta refresh on, Mechanize will wait 30 mins to load the meta refresh, and will continually do this until the redirect limit is reached.

For my purposes, I don't need to wait for the delay; I just want to know if the page is going to meta-redirect within a certain window and to where (and if the redirected page will do the same).

[flavorjones]

I agree, it would be nice to be able to modify this behavior. See #429 and #523 for examples of the confusion that can ensue.