• whaleross@lemmy.world
        link
        fedilink
        arrow-up
        3
        ·
        1 year ago

        There is probably a neat wget oneliner that could crawl everything on the open web. The real challenge is how to index all the information. That might be a neat Perl oneliner.