05 October 2014

Checking your website links during development

When creating a website, it can become very hard to check if all the links you created are valid and working. I know you may never make such errors, but some people do ;-)
Or, you're maintaining your website and you make a few changes that break some links. How do you check for these? You couldn't expect to manually click on each and every link....

Two tools come in handy here: Xenu and Windows Grep (or use it in Linux using Wine and grep).
Windows Grep is handy in case you want to grep any file names and locate the calling file. Besides, Windows Grep also supports regex. Linkchecker can also show you the calling file, but Xenu beats it hand's down, in terms of features.


Xenu in action

Give Xenu a web link or even point it to the index page of a website stored on your PC. Xenu shows you broken links, empty web pages, missing files, the depth of search (enter a link and move on to another link on that page and so on...), the number of links outgoing from a page, the number of links coming into a page, the error and the character-set. Right click on a row and you get options to show which file the report belongs to.
Under the covers, it's doing a graph traversal. Same way that the search engine bots crawl the internet.


Xenu even converts URL's to lower case if you wish (in Windows, www.Website.com is the same as www.website.com. But to Linux, they're different).


Crawling through links is assigned to multiple threads (although the real number of physical threads will be equal to the number of hardware threads in your processor. The extra threads will have to do some time-sharing).



Compared to all this, Linkchecker has a very limited functionality:



More

There are even commercial link checkers with more advanced functionality than Xenu. Some are:


No comments: