https question

[Everton]

okay, it's been a while since I've asked any dumb simple questions you all know the answers to.

when a site is using https instead of http, can spiders still crawl that site?

My understanding is those pages load slightly slower and are just more secure. But I'm starting to suspect they cause all sorts of problems with spiders.

Thoughts?

[Andy]

I guess the spiders are attracted to the "s" that is like the "S" in spider  It's close to the start of the URL so should really help with ranking. Just an idea 

Oops, I forgot to mention that the https pages are scrambled so won't be indexed, hence my suggestion to focus on the "s" in the url.

[White Wolf]

I just put up a site using https and most pages are now indexed.

[Hope]

Why would you have an entire site in HTTPS?

[Andy]

It suprises me that the pages are indexed since they are likely to be pages by banks and shopping carts etc.

[White Wolf]

Why would you have an entire site in HTTPS?

Because Users can connect to a demo admin area or their control panel from the home page. I also collect CC details.


Each page can be accessed by http if I type it in.
I think spiders still see the same content it only encrypt the data that that a user fills in and passes it to somewere else.

[donecweb]

Sounds like having your test sites located in https would be a good way to keep them from being shown in search engines since they are scrambled. At what point are they scrambled? I often miss things that are obvious to others so is there something I am missing here?

[White Wolf]

At what point are they scrambled?

The page code it self is not scrambled. check source of any https page what ever you see in that souce a spider can see as well.

When a user fills out a form with their info that info is not part of the page source. When filling out a form your info has to be passed from point A  to point B  If your site is not secure that info can be intercepted before it reaches point B

If a user put in their info on a sucure site their info passes from A-B encrypted  so even if it is intercepted before reaching B it can not be read.
 

[White Wolf]

I just typed in secure web store in google quit a few https sites are indexed

http://www.google.com/search?q=secure+web+store&hl=en&rls=com.microsoft:en-us:IE-SearchBox&rlz=1I7GGIH&start=0&sa=N

[donecweb]

Thanks that makes more sense.

[White Wolf]

after doing some more reading on the issue I found that i'm not totally right...google and yahoo can Index https pages but MSN the third largest searh engine IMHO can't or wont index https pages. They said that they will be adding support for HTTPS pages in the future.

I guess I got to go back and change my most of my links to http. I can always just secure the login box on my index by
onsubmit-->https://  and set other pages in cart to https

EDIT I just realised the post I was reading from "MSNDUDE" was from August 2005 after checking the MSN search results they are now listing https pages.

I'm still going to change all thats possable to http To make sure my pages can be spidered by all engines.