Search
-
Add a Web Crawler System to walk/scrape multiple pages · planned
-
Store results in S3 · planned
-
multithreaded
-
Manage rate-limiting internally · declined
-
Please allow to post data in url as url parameters. · completed
-
Add fine control of JSON output verbosity · completed
-
Allow modifying the http headers being sent back from the page response. · completed