User-agent: * Disallow: /user/ Disallow: /item/ Disallow: /advertiser/registrations/ Disallow: /advertiser_registrations/ Disallow: /publisher/registrations/ Disallow: /publisher_registrations/