Robots.txt

Robots.txt

A file placed on a web server that gives instructions to search engine crawlers, which are the robots that index web pages for search engines. Robots.txt files tell web crawlers what they should include in their index—and what they should ignore. Pages such as form results pages (“thank you” pages that are triggered when users submit a form, for example) can be hidden from search engines.

    • Related Articles

    • Marketing Automation

      A process that enables technology to take over repetitive marketing tasks from people, freeing people to focus on strategy. Technology can automate scheduled email sends and social site postings. It can determine clear winners among deployed ...

    • Direct Marketing

      Marketing that is delivered directly to the customer via the company selling a product. It’s often called direct response marketing. It typically includes a call to action (CTA) requesting a desired response such as calling a phone number or visiting ...

    • Digital Marketing

      Any marketing that uses electronic devices to convey promotional messaging and measure its impact. In practice, digital marketing typically refers to marketing campaigns that appear on a computer, phone, tablet, or other device. It can take many ...

    • Integrated Marketing

      The practice of aligning all marketing tactics to the same core messaging for a consistent customer experience with your brand. Through integrated marketing, communications tactics such as display ads, landing pages, email marketing, direct mail, and ...

    • Inbound Marketing

      A marketing strategy that encourages people to actively seek out and engage with your brand. Social media, content marketing (such as sponsored content and blogs), and search engine optimization (SEO) are common tactics. Inbound marketing offers ...