Robots.txt Disallow Urls

[White Wolf]

My site uses somekind of redirect

The urls look like " www. mydomain.com/?page=apply&id=XXXX     were XXXX displays the item ID #

My question is there a way to disallow  just  /?page=apply&id=XXXX ?  with one line or do i need to add each url

User-agent: *
Disallow:  /?page=apply&id=1234
Disallow:  /?page=apply&id=1235
Disallow:  /?page=apply&id=1236


   

[Andy]

You only need part of the url e.g.

Disallow:  /?page=apply


From http://www.robotstxt.org/wc/norobots.html :

Disallow
    The value of this field specifies a partial URL that is not to be visited. This can be a full path, or a partial path; any URL that starts with this value will not be retrieved. For example, Disallow: /help disallows both /help.html and /help/index.html, whereas Disallow: /help/ would disallow /help/index.html but allow /help.html

[White Wolf]

Thanks for your reply Andy.

I'm tired of having to add to the Robots.txt everytime I add a card.

[SensoVision]

Kevin, to make sure you've banned right pages you may use a tool which comes with Google Webmaster Central: http://www.google.com/webmasters/sitemaps/siteoverview
once googlebot spider your page you can simply check certain ulrs if they are accessible for spider. There are other useful tools available there so it's worth to register.

[White Wolf]

Thanks Dennis!

I tried out the tool, and Andy's suggestion does work correctly.