I was recently looking for broken links on my websites using ahrefs tool, but it kept asking for money. I realized how easy what they were doing is to implement, so I did it myself.

That's how I made crawlr. It will recursively search all of the urls on your website save them into a csv file with their status code and let you know how many urls are broken.

It is written in go and is extremely fast. It takes a couple of seconds to index a couple hundered of urls.

Give it a ⭐ if you like it