We are likely to rate limit this reader to our usual 400 requests per 5 minutes

[Discordius]

Hey,

So currently turns out this reader is making up around 10-20% of our traffic, and tends to be skewed towards somewhat more expensive queries, so that it's actually kind of costing us real money (and costs in reliability) to serve all the requests.

We had deactivated rate-limiting for the IP address of your server, but my guess is we are going to undo that, unless there are a bunch of people who use this actively, or it would be quite costly for you to somehow reduce the number of requests (compare with GW which seems to make many fewer requests and doesn't have this problem).

[riceissa]

That's fair, and I am very sorry about that! I've noticed a recent increase in bot traffic on one of my other sites, so I am guessing that's what's going on. I will look into figuring out how to reduce the number of requests but can't give a definite timeline on that. In the meantime, please go ahead with re-activating the rate-limiting. I appreciate the notice.

[riceissa]

I did an initial round of work to rate-limit and block a bunch of bot requests, so traffic coming from my server should now be more reasonable. I am guessing very few people use my reader so 400 requests per 5 minutes still seems like a fine limit to have in place as a safety measure in case the bots get around my current filters.