Transferring files between servers?

[Menard]

On one of my hosting accounts, I would like to transfer some reseller content to it from another server. I have download access to the files, but downloading and uploading files collectively over 8GB would take a very long time. I was wondering if someone knew of a way, or software, that would let me transfer files from a link and into my hosting account?

[SensoVision]

If your server have SSH access than it's very easy. You can make steps similar to these:
on server from which you need to download file simply create archive of data need to be transferred like this:
tar jcvf archive.tar.bz2 somedata-directory
place archive.tar.bz2 in the place where you can download it through http or FTP.

login into SSH on other server and simply download it to some directory:
it could be done like this:
wget -c http://yoursiteurl.net/archive.tar.bz2
now just unpack data and it would be ready for use:
tar jxvf archive.tar.bz2

If you decide to transfer files this way I advice creating smaller archives than 1Gb, e.g. 600mb or so, this way it would be easier to transfer file again if something goes wrong and you wouldn't waste as much bandwidth as you could with bigger files. And I remember that in the past there was limitation of wget for 2gb files or so but it was overcome in later versions.(If wget isn't installed you may try curl)

I'm sure there are GUI interfaces which let you download files, but I'm not aware of them. There is also way to write your own but I'm not sure how PHP would work with such big files.
BTW if you don't have SSH enabled on the new server, you may simply ask admins to download files for you from your old server by providing them with list of files to be downloaded.

Let me know if you need more help.

[Andy]

I posted about this exact same topic recently. The web-server-based software I used only allowed 10 files at a time to be transferred.

A small script would be ideal but I don't know of one.

You have to be aware that your server will have a maximum file size limit of maybe 20MB. Check it with PHPinfo.php

[SensoVision]

You have to be aware that your server will have a maximum file size limit of maybe 20MB. Check it with PHPinfo.php

that's right, there is also timeout parameter(which determine time how long one script could run) and it may limit transfer as well, and these options usually(in rare cases user may control them) set by hosting administrator and are not likely to be altered.

[Andy]

It just struck me that this is one good reason to get hold of your own dedicated server, then you can host unlimited sites and easily transfer large chunks of data between the sites since the data transfer will be done on the same server and would be very fast. Also, you should get SSH access to allow very good control of your server.

Transferring Gigs of data up and down from the web and your local computer is a bad idea due to speed and hosting bandwidth limits.

[SensoVision]

I'm 100% agree with you, Andy! the only drawback of hosting all sites on the same server is that they could go offline simultaneously and make hacker job easier...
But having SSH for administration is must have tool for any developer or to webmasters who have at least two sites or one with data which need regular backup.

[Andy]

Well I have survived with shared virtual server hosting for a long time at around $20 month. What do you pay for a dedicated server? Actually my son runs a server from his room so I understand that there are big differences in the quality of dedicated servers from an old dusty 386 PC in a garage up to a rack mounted machine in an air-conditioned data center.

[Menard]

I had once used an online FTP service to transfer files between hosting accounts. There is a free online service setup specifically for that, but they have file size limits as well.

What I am needing to transfer, though, is from a link to a hosting account, not from account to account.


Right now, I'm thinking my best options are one of two:

Contacting support to see if someone would transfer the files for me (they are password protected and some are over 800MB)

or

Downloading all of the files myself (my download speed is like 6-7 times faster than my upload speed) then borrowing a friend's connection at work to upload the files (I could feasibly download the files there as well as they have fast connections). I would just have to find the time to do that.


I would eventually like to have a dedicated server. Of course, two things stand in the way; I can't afford it, and I have no place to put one if I were to use my own. With what I have planned for the new site, I am either going to need dedicated hosting, or some very understanding shared hosting.

[Andy]

I am either going to need dedicated hosting, or some very understanding shared hosting.

Shared hosting is fine but when you hit the bandwidth limit, it's like a brick wall, your hosting is suspended until the next month. A couple of my sites run on unlimited bandwidth hosting but I don't believe it is. Better to pay for a known limit of bandwidth I think.

borrowing a friend's connection at work to upload the files (I could feasibly download the files there as well as they have fast connections).

Bad idea! The IT department monitor all internet traffic and maybe produce reports to management so your friend could get sacked or disciplined for inappropriate use of company resources. Better to fly under the radar with Internet use at work. Doing anything unusual rings alarm bells. For example, I can never understand employees using their work email for personal use. I always used web-based email.

p.s. How about using a cyber cafe with fast internet connections? For a small fee you could maybe do the transfers using their fast link?

Also, I don't know if you could make use of a file hosting service like http://www.mediafire.com/ ?

[Softix]

I have read that hostgator is doing free transfer to the new hosts. I got some problems before in my hosting and it seems that they offer such great services.

[Queen Bee]

I have read that hostgator is doing free transfer to the new hosts. I got some problems before in my hosting and it seems that they offer such great services.

I was happy to see that as well. I have been very disappointed my hosting service in recent months (I use Hostexcellence.) With the advice of Andy I'm going to try out Hostgator and post my experience here.