22 commits

Author	SHA1	Message	Date
messense	553b47f7e7	Change from_response method signature	2016-08-25 18:01:17 +08:00
Ernestas Poskus	36c90ec169	Add method read_from_response	2016-08-25 09:59:35 +03:00
messense	17a36159a7	Add Request-rate support. Closes #4	2016-08-21 11:35:24 +08:00
messense	a5254dbc52	style: rustfmt source codes	2016-08-21 10:58:18 +08:00
Laurent Arnoud	e3663eb257	Use std::time instead of deprecated crate time	2016-08-20 20:24:40 +02:00
Laurent Arnoud	8b50c90f94	Set UserAgent as robotparser-rs	2016-05-04 23:43:32 +02:00
messense	caca158589	Use Cow instead of owned String	2016-04-27 11:26:28 +08:00
messense	a552d99210	Do not deny all clippy issues ... Signed-off-by: messense <messense@icloud.com>	2016-04-25 13:23:01 +08:00
messense	77b604d2af	Bump version to 0.4.3 ... Signed-off-by: messense <messense@icloud.com>	2016-04-25 12:33:15 +08:00
messense	74fddfd5cb	Add implementation of Default for Entry	2016-03-17 13:29:58 +08:00
messense	7f9d08b307	Better documentation	2016-03-06 16:31:44 +08:00
messense	2d9c028dfa	Refactor codes like clippy suggested	2016-03-06 16:19:58 +08:00
Mikhail Svetov	fcfb6d9df6	Improvements.	2016-02-08 17:25:27 +07:00
Mikhail Svetov	626fa947e4	Added sitemap support.	2016-02-08 17:23:46 +07:00
Mikhail Svetov	02eedd6246	Improvements.	2016-01-13 21:18:07 +07:00
Mikhail Svetov	49a2b9c93d	Added crawl delay feature.	2016-01-13 20:51:54 +07:00
messense	ac7d7aa8db	Refactor some API to be more generic over Rust strings	2015-10-21 15:40:06 +08:00
messense	160ed940ae	Use join instead of connect	2015-08-12 17:53:28 +08:00
messense	8e2944d8a4	Bug fixes and add more test cases	2015-07-01 00:01:15 +08:00
messense	e8928ec198	Bug fix and add more tests	2015-06-28 09:59:52 +08:00
messense	6153773f24	Correct the usage of RefCell.borrow	2015-06-28 01:39:03 +08:00
messense	ece1e1f3c4	Initial commit.	2015-06-28 01:29:05 +08:00