v0.6.0

• added fast-h2m library to convert HTML to Markdown
• upgraded scraper-rust to v0.6.0 (drasdically enhances speed of HTML selectors)

v0.5.3

• add Lean 4 formal model for Silkworm
• fixed issues found by formal model
• add a middleware to control crawler using values from robots.txt
• add an example start_urls_from_file_spider.py

v0.5.2

• add a middleware to control cookies

v0.5.1

• added Servo browser to get pages with OnionLink integration (parse TOR websites)
• enhanced scraper-rs to use all cores for async parsing
• added errback support (resolves #163)
• added logger engine to control how much logs generated (resolves #177)
• upgraded other deps

v0.4.8

• integrated async client for onionlink

v0.4.7

• add onionlink to scrape TOR websites

v0.4.6

fix #169

v0.4.5

• fix #170

v0.4.4

upgrade scraper-rust

v0.4.3

update rsloop