For publishers

Kliper's crawler

What KliperMedia fetches, why, how politely, and how to reach us.

Kliper is a media monitoring service. Our crawler identifies itself with this User-Agent:

KliperMedia/1.0 (+https://klipermedia.com.br/bot)

What it fetches and why

We read news feeds, news sitemaps and article pages to tell our clients when they are mentioned in the news.

No model training

Kliper does not train its own models on your content.

robots.txt and Crawl-delay

When we look for a site's feed, we first read its robots.txt and follow its rules for KliperMedia and for *, including a Crawl-delay of up to 60 seconds.

Request rate

When we fetch from a site, each of our processes waits at least 1 second between two requests to it; the hops of one redirect count as one request.

Across all sites, we fetch at most 600 article pages every 10 minutes per capture process.

Where our requests come from

Write to us to confirm that a request came from Kliper.

Contact and opt-out

To ask us to slow down or to stop fetching from your site, write to: contato@klipermedia.com.br

We stop fetching from a site within two business days of its request.