php - Automated URL checking from a MySQL table -
okay, have list of urls in mysql table. want script automatically check each link in table 404, , afterward want store whether url 404'd or not, store time last checked.
is possible automatically, if no 1 runs script? ie, no 1 visits page few days, no 1 visiting page, automatically ran test.
if possible, how go making button this?
no need use curl, file_get_contents($url);
return false if request fails (any other http code other 2xx), might more useful you're trying do, example:
function urlexists($url) { return (bool) @file_get_contents($url); }
will return true if url returns useful content, false otherwise.
edit: here faster way (it requests headers) , the first byte instead of whole page:
function urlexists($url) { return (bool) @file_get_contents($url, false, null, 0, 1); } urlexists('https://stackoverflow.com/idontexist'); // false
however, in combination your other question may wiser use this:
function url($url) { return @file_get_contents($url); } $content = url('https://stackoverflow.com/'); // request has failed (404, 5xx, etc...) if ($content === false) { // delete or store "failed" in db } // request successful else { $hash = md5($content); // md5() should enough can use sha1() // store $hash in db keep track of changes }
or if you're using php 5.1+ have do:
$hash = @md5_file($url);
$hash
false when url fails load, otherwise return md5 hash of contents.
graciously stolen @jamie. =)
this way have make 1 request instead of two. =)
Comments
Post a Comment