Filename: /var//www/arsu//vendor///spatie//crawler///CHANGELOG.md

# Changelog All notable changes to `spatie/crawler` will be documented in this file. ## 6.0.1 - 2021-02-26 - bugfix: infinite loops when a CrawlProfile prevents crawling (#358) ## 6.0.0 - 2020-12-02 - add `setCurrentCrawlLimit` and `setTotalCrawlLimit` - internal refactors ## 5.0.2 - 2020-11-27 - add support for PHP 8.0 ## 5.0.1 - 2020-10-09 - tweak variable naming in `ArrayCrawlQueue` (#326) ## 5.0.0 - 2020-09-29 - improve chucked reading of response - move observer / profiles / queues to separate namespaces - typehint all the things - use laravel/collections instead of tightenco package - remove support for anything below PHP 7.4 - remove all deprecated functions and classes ## 4.7.5 - 2020-09-12 - treat connection exceptions as request exceptions ## 4.7.4 - 2020-07-15 - fix: method and property name error (#311) ## 4.7.3 - 2020-07-15 - add crawler option to allow crawl links with rel="nofollow" (#310) ## 4.7.2 - 2020-05-06 - only crawl links that are completely parsed ## 4.7.1 - 2020-04-14 - fix curl streaming responses (#295) ## 4.7.0 - 2020-04-14 - add `setParseableMimeTypes()` (#293) ## 4.6.9 - 2020-04-11 - fix LinkAdder not receiving the updated DOM (#292) ## 4.6.8 - 2020-03-12 - allow tightenco/collect 7 (#282) ## 4.6.7 - 2020-03-09 - respect maximum response size when checking Robots Meta tags (#281) ## 4.6.6 - 2020-01-30 - allow Guzzle 7 ## 4.6.5 - 2019-11-23 - allow symfony 5 components ## 4.6.4 - 2019-09-10 - allow tightenco/collect 6.0 and up (#261) ## 4.6.3 - 2019-09-09 - fix crash when `CrawlRequestFailed` receives an exception other than `RequestException` ## 4.6.2 - 2019-08-08 - case-insensitive user agent bugfix (#249) ## 4.6.1 - 2019-08-08 - fix bugs in `hasAlreadyBeenProcessed` ## 4.6.0 - 2019-08-07 **THIS VERSION CONTAINS A CRITICAL BUG, DO NOT USE** - added `ArrayCrawlQueue`; this is now the default queue - deprecated `CollectionCrawlQueue` ## 4.5.0 - 2019-07-22 - Make user agent configurable (#246) ## 4.4.3 - 2019-06-22 - `delayBetweenRequests` now uses `int` instead of `float` everywhere ## 4.4.2 - 2019-06-20 - remove incorrect docblock ## 4.4.1 - 2019-06-06 - handle relative paths after redirects correctly ## 4.4.0 - 2019-04-05 - add `getUrls` and `getPendingUrls` ## 4.3.2 - 2019-04-04 - Respect maximumDepth in combination with robots (#181) ## 4.3.1 - 2019-04-03 - Properly handle `noindex,follow` urls. ## 4.3.0 - 2019-03-11 - added capability of crawling links with rel= next or prev ## 4.2.0 - 2018-10-31 - add `setDelayBetweenRequests` ## 4.1.7 - 2018-07-27 - fix an issue where the node in the depthtree could be null ## 4.1.6 - 2018-06-26 - improve performance by only building the depth three when needed - handlers will get html after JavaScript has been processed ## 4.1.5 - 2018-06-25 - refactor to improve extendability ## 4.1.4 - 2018-06-09 - always add links to pool if robots shouldn't be respected ## 4.1.3 - 2018-06-05 - refactor of internals ## 4.1.2 - 2018-06-03 - make it possible to override `$defaultClientOptions` ## 4.1.1 - 2018-05-22 - Bump minimum required version of `spatie/robots-txt` to `1.0.1`. ## 4.1.0 - 2018-05-08 - Respect robots.txt ## 4.0.5 - 2018-04-30 - improved extensibility by removing php native type hinting of url, queue and crawler pool Closures ## 4.0.4 - 2018-03-20 - do not follow links that have attribute `rel` set to `nofollow` ## 4.0.3 - 2018-03-02 - Support both `Illuminate`'s and `Tighten`'s `Collection`. ## 4.0.2 - 2018-03-01 - fix bugs when installing into a Laravel app ## 4.0.0 - 2018-03-01 - the `CrawlObserver` and `CrawlProfile` are upgraded from interfaces to abstract classes - don't crawl `tel:` links ## 3.2.1 - 2018-02-21 - fix endless loop ## 3.2.0 - 2018-01-25 - add `setCrawlObservers`, `addCrawlObserver` ## 3.1.3 - 2018-01-19 - fix `setMaximumResponseSize` (someday we'll get this right) ## 3.1.2 - 2018-01-19 **CONTAINS BUGS, DO NOT USE THIS VERSION** - fix `setMaximumResponseSize` ## 3.1.1 - 2018-01-17 **CONTAINS BUGS, DO NOT USE THIS VERSION** - fix `setMaximumResponseSize` ## 3.1.0 - 2018-01-11 **CONTAINS BUGS, DO NOT USE THIS VERSION** - add `setMaximumResponseSize` ## 3.0.1 - 2018-01-02 - fix for exception being thrown when encountering a malformatted url ## 3.0.0 - 2017-12-22 - use `\Psr\Http\Message\UriInterface` for all urls - use Puppeteer - drop support from PHP 7.0 ## 2.7.1 - 2017-12-13 - allow symfony 4 crawler ## 2.7.0 - 2017-12-10 - added the ability to change the crawl queue ## 2.6.2 - 2017-12-10 - more performance improvements ## 2.6.1 - 2017-12-10 - performance improvements ## 2.6.0 - 2017-10-16 - add `CrawlSubdomains` profile ## 2.5.0 - 2017-09-27 - add crawl count limit ## 2.4.0 - 2017-09-21 - add depth limit ## 2.3.0 - 2017-09-21 - add JavaScript execution ## 2.2.1 - 2017-09-07 - fix deps for PHP 7.2 ## 2.2.0 - 2017-08-03 - add `EmptyCrawlObserver` ## 2.1.2 - 2017-03-06 - refactor to make use of Symfony Crawler's `link` function ## 2.1.1 - 2017-03-03 - fix bugs around relative urls ## 2.1.0 - 2017-01-27 - add `CrawlInternalUrls` ## 2.0.7 - 2016-12-30 - make sure the passed client options are being used ## 2.0.6 - 2016-12-15 - second attempt to fix detection of redirects ## 2.0.5 - 2016-12-15 - fix detection of redirects ## 2.0.4 - 2016-12-15 - fix the default timeout of 5 seconds ## 2.0.3 - 2016-12-13 - set a default timeout of 5 seconds ## 2.0.2 - 2016-12-05 - fix for non responding hosts ## 2.0.1 - 2016-12-05 - fix for the accidental crawling of mailto-links ## 2.0.0 - 2016-12-05 - improve performance by concurrent crawling - make it possible to determine on which url a url was found ## 1.3.1 - 2015-09-13 - Ignore `tel:` links when crawling ## 1.3.0 - 2015-08-18 - Added `path`, `segment` and `segments` functions to `Url` ## 1.2.3 - 2015-08-12 - Updated the required version of Guzzle to a secure version ## 1.2.2 - 2015-03-08 - Fixed a bug where the crawler would not take query strings into account ## 1.2.1 - 2015-03-01 - Fixed a bug where the crawler tries to follow JavaScript links ## 1.2.0 - 2015-12-21 - Add support for DomCrawler 3.x ## 1.1.1 - 2015-11-16 - Fix for normalizing relative links when using non-80 ports ## 1.1.0 - 2015-11-16 - Add support for custom ports ## 1.0.2 - 2015-11-05 - Lower required php version to 5.5 ## 1.0.1 - 2015-11-03 - Make url's case sensitive ## 1.0.0 - 2015-11-03 - First release

./Ninja\.