Site Scraping Terms

Scope

These Site Scraping Terms ("Scraping Terms") govern all automated access to marketxy.com and its subdomains — including crawlers, spiders, scrapers, headless browsers, scripted HTTP clients, and any other non-human agent that requests pages or assets from this site.

They sit alongside our Terms of Use and Privacy Policy. Where a conflict arises with respect to automated access, these Scraping Terms control.

These Scraping Terms govern access to this website. They do not restrict use of data you have licensed from us through the API, a bulk database purchase, or a data feed — that use is governed by the Terms of Use and your order form.

Prohibited Automated Collection

Collection of data from marketxy.com through automated means is prohibited unless you have express written permission from MarketXY, and may only be conducted for the limited purpose contained in that permission. Without such permission you may not:

• Crawl, scrape, harvest, or index page content, search results, lookup results, or listing pages for republication, resale, or inclusion in a competing dataset or product.
• Circumvent, disable, or interfere with rate limits, bot detection, CAPTCHA challenges, authentication, or access controls.
• Misrepresent your identity — including forging a User-Agent to impersonate a browser or a permitted crawler, or rotating IPs or accounts to evade limits.
• Use automated access to extract personal data at scale from WHOIS records, or to build or enrich marketing, spam, or contact lists.
• Place a load on the site that degrades service for other users.

Permitted Automated Access

The following automated access is permitted without separate written permission:

• Search-engine and AI crawlers that identify themselves honestly and obey robots.txt, for the purpose of indexing this site and surfacing it to their users.
• Availability and security monitoring of a small number of pages at a low request rate.
• Our own API, under your API key and plan limits. The API — not this website — is the supported way to obtain WHOIS data programmatically; see the API documentation.

Permitted crawlers must still respect the directives in robots.txt, which is authoritative for which paths may be crawled. Some namespaces — the unbounded, search-driven lookup pages such as /domain/<name>/ — are disallowed there because they are generated per query rather than published as content.

Rate Limits and Crawl Etiquette

Where automated access is permitted, it must be conducted courteously:

• Send a descriptive User-Agent that names the crawler and a URL or email address we can use to reach its operator.
• Observe any Crawl-delay in robots.txt; absent one, keep to no more than one request per second per host and no more than two concurrent connections.
• Back off on 429 or 5xx responses rather than retrying immediately, and honour Retry-After.
• Cache responses and use conditional requests (If-Modified-Since, If-None-Match) rather than re-fetching unchanged pages.

Personal Data in WHOIS Records

WHOIS records can contain personal data. Automated collection of that data from this site is subject to the GDPR, the CCPA, and comparable laws, and bulk extraction for marketing or unsolicited contact is prohibited outright — irrespective of any permission granted for other purposes.

If your data appears in our index and you want it removed, use the data removal request form rather than a technical workaround. We process requests within 30 days.

Requesting Permission

Bulk or ongoing automated access is normally better served by a licensed data product than by crawling this website — the API and bulk downloads are faster, structured, and come with a licence that says what you may do with the data.

To request written permission for automated collection, or to discuss a licence, use the contact form and describe the data you need, your intended purpose, your expected request volume, and the User-Agent your crawler will send.

Enforcement

We may rate-limit, challenge, or block any client we believe is in breach of these Scraping Terms, at the network or account level and without prior notice. Breach may also result in suspension or termination of an account and any licence granted under it.

We reserve all rights and remedies available to us in law, including under applicable computer-misuse, contract, and database-rights legislation.

Changes to These Terms

We may update these Scraping Terms. Material changes will be reflected in the "last updated" date at the top of this page. Continued automated access after a change constitutes acceptance of the revised Scraping Terms.