dan/robotparser-rs
1
0
Fork 0
mirror of https://github.com/danbulant/robotparser-rs synced 2026-05-19 04:18:42 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
106 commits 1 branch 0 tags 185 KiB
Commit graph

106 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Bulant
165de042c4
update reqwest version 2025-08-31 14:23:32 +02:00
Daniel Bulant
e889cf1855
fix reqwest feature being mandatory 2025-08-31 14:21:14 +02:00
messense
ec7b2fd986
Fix some more clippy warnings 2022-03-30 10:21:52 +08:00
messense
6db945c28f
Fix a clippy warning 2022-03-30 10:18:13 +08:00
messense
548ea65072
Update reqwest to 0.11 2022-03-30 10:15:35 +08:00
messense
70384a5fbf Remove CI cache 2020-11-03 19:05:43 +08:00
Maxim Zhukov
abcb18dc97
Add support coverage via codecov + tarpaulin (#31) 
* workflows: add support coverage

Signed-off-by: Maxim Zhukov <mussitantesmortem@gmail.com>

* workflows: use HOME instead of '~' for fix macos cache build

Signed-off-by: Maxim Zhukov <mussitantesmortem@gmail.com>
 2020-11-03 19:04:47 +08:00
Maxim Zhukov
230c4201ca
cargo: update (#30) 
Signed-off-by: Maxim Zhukov <mussitantesmortem@gmail.com>
 2020-11-03 18:26:28 +08:00
Maxim Zhukov
e7b82b7079
parser: replace manual strip with trim_start_matches (#29) 
Signed-off-by: Maxim Zhukov <mussitantesmortem@gmail.com>
 2020-11-03 18:26:14 +08:00
Maxim Zhukov
9bb7583f30
README: remove old rust instructions (#28) 
Signed-off-by: Maxim Zhukov <mussitantesmortem@gmail.com>
 2020-11-03 18:25:55 +08:00
messense
408f792ebd Commit Cargo.lock to git for cache to actually work 2020-03-30 09:42:35 +08:00
messense
5861623b6b Rename workflow name to CI 2020-03-30 09:39:46 +08:00
messense
85c108ef43 Remove Travis & Appveyor 2020-03-30 09:36:57 +08:00
Laurent Arnoud
6ba403aab9
Inital github-actions (#25) 
* Inital github-actions

most taken from starship project
ref #21

* rustfmt config remove unknown configuration options

* Run rustfmt

* clippy: use any instead of find..is_some

* clippy: Remove the `clone` call: `self.crawl_delay`

* Clippy fixes

* Rustfmt fixes

* clippy: fix dont need to add `&` to all patterns

* clippy: fix needless `fn main` in doctest

* clippy: fix if-then-else expression returns a bool literal

* clippy: fix very complex type BoxFuture response

* clippy: fix variable `line_no` is used as a loop counter

* clippy: dereference the expression on tests

* clippy: fix assert(true) will be optimized out by the compiler

* github: name workflow
 2020-03-30 09:32:32 +08:00
Laurent Arnoud
df49f6bcf0
Error handling (#24) 
* Add test when url is invalid and panic

* Initial error handling

ref https://github.com/messense/robotparser-rs/issues/22

* Rename ErrorKind::HttpClient => ErrorKind::Http

* Implement std::error::Error and rename to Error
 2020-03-08 20:33:34 +08:00
Laurent Arnoud
1474a8cce9
Switch to https for urls (#23) 2020-03-02 15:20:44 +08:00
svmk
2d19755779
Refactoring of robotparser-rs (#20) 
* Migrated sites into robotsparser file.

* Robots.txt refactoring.

* Migrated to new version of url and reqwest.
 2020-01-31 17:00:58 +08:00
messense
cb7df85b83 Bump version to 0.10.2 2018-09-22 11:47:15 +08:00
messense
86a483be75 Use openssl 1.0.2p on Appveyor 2018-09-22 11:44:43 +08:00
messense
2205ef3b5d Remove clippy for now 2018-09-22 11:33:28 +08:00
Laurent Arnoud
13e60c226a Update reqwest to 0.9 (#16) 2018-09-22 11:30:41 +08:00
messense
ed0f892875
Use openssl 1.0.2o open AppVeyor 2018-04-02 13:17:33 +08:00
Laurent Arnoud
141109aa15 Add deps status badge (#15) 2018-04-02 10:31:34 +08:00
messense
9d46f5ac99
Bump version to 0.10.1 2018-03-12 09:29:22 +08:00
messense
4688c72346
Merge pull request #14 from spk/fix-127.0.0.1-host-panic 
Use host_str instead domain for HostInternal
 2018-03-12 09:23:09 +08:00
Laurent Arnoud
19bd3b4cf6
Use host_str instead domain for HostInternal 2018-03-11 22:05:14 +01:00
messense
953d1f0c0b
Again fix AppVeyor 2018-01-11 13:43:24 +08:00
messense
6854290e18
Use openssl 1.0.2n on Appveyor 2018-01-11 12:31:40 +08:00
messense
6adf740663
Use url crate 1.x version 2018-01-11 11:28:52 +08:00
messense
d6127d25bb
Bump version to 0.10.0 2017-12-02 23:52:48 +08:00
messense
5ad2785e07
Fix some clippy warnings 2017-12-02 23:51:52 +08:00
messense
3b24290e5c
Appveyor use openssl 1.0.2m 2017-12-02 23:49:41 +08:00
messense
450cafadfd
Merge pull request #13 from spk/update-request 
Update reqwest to 0.8
 2017-12-02 23:47:59 +08:00
Laurent Arnoud
79471efcc2
Update reqwest to 0.8 2017-12-02 16:39:25 +01:00
messense
832eb63e36
Update doc refs version 2017-08-09 13:21:28 +08:00
messense
8fdbd92042
Bump version to 0.9.0 2017-08-09 13:17:06 +08:00
messense
44eb658238
Upgrade reqwest to 0.7 2017-08-09 13:16:27 +08:00
messense
63016cd97a
Update version to 0.8 in docs 2017-04-02 00:08:43 +08:00
messense
9f57b38e02
Bump version to 0.8.2 2017-04-02 00:05:43 +08:00
messense
b4b0752e96
Upgrade reqwest to 0.5 2017-04-02 00:05:16 +08:00
messense
cd4d32bf28
Bump version to 0.8.1 2017-03-13 23:10:57 +08:00
messense
b27c5544eb Merge pull request #12 from spk/less-strict-reqwest-vers 
Less strict version on reqwest
 2017-03-12 22:11:27 +08:00
messense
426365bc9d
Set RUST_BACKTRACE=1 on Travis CI 2017-03-12 22:10:33 +08:00
Laurent Arnoud
be7f58adec
Less strict version on reqwest 
Using MINOR from semver
 2017-03-12 15:03:38 +01:00
messense
101f926adc
Bump version to 0.8.0 2016-11-14 16:14:40 +08:00
messense
e3b9ca6c6b Merge pull request #11 from spk/use-request-http-client 
Use request higher level Client crate for openssl updates
 2016-11-14 16:10:18 +08:00
Laurent Arnoud
9f5b4c27d4
Use request higher level Client crate for openssl updates 
see https://github.com/hyperium/hyper/issues/907#issuecomment-255509020
 2016-11-12 18:22:55 +01:00
messense
a94a5b34cc
Replace 0.6 with 0.7 in README and docstring 2016-10-21 14:39:08 +08:00
messense
29c009114c
Bump version to 0.7.0 2016-10-05 13:30:10 +08:00
messense
760a4e723b Merge pull request #10 from messense/feature/feature-gate-hyper 
Feature gate hyper
 2016-10-05 13:28:45 +08:00