URLFrontier update

0 views

Skip to first unread message

Julien Nioche

unread,

Jul 30, 2026, 5:35:24 AMJul 30

to crawler...@googlegroups.com

Hi,

There has been a renewed interest in URLFrontier lately and many improvements have been made to the code (thanks to AI in parts). Apache StormCrawler now makes better use of the flexibility that URLFrontier provides. There are quite a few things flagged for the next release but the code is already much more robust and way faster than the version 2.5!

For those of you who use it, could you please vote on https://github.com/crawler-commons/url-frontier/discussions/196 ?

I am about to disappear for two weeks, hopefully we will push a new release before September. If you haven't used URLFrontier recently, why not give it a try?

Enjoy the rest of the summer

Julien Nioche

Founder & Director