Migrating from Shared Hosting to WPEngine the Fast Way

May 22, 2013

 

In the last post, we sort of laughed about how we’ve rescued so many publishers from the grips of OpenX. But there’s another villain in the land of hyperlocal publishing technology: the hosting provider.

The most popular web hosts among our publishers appear to be Dreamhost, HostGator, and Bluehost. Now, we won’t go pointing fingers, but at least one of these companies can’t keep it together.

It’s not exactly our core business, but we do occasionally help our publishers migrate from one server to another (we especially like it when they plan to use our adserver afterward!).

Here’s a detailed migration guide that will help your tech person migrate the most gnarly of sites from a shared host or lousy VPS to WPEngine in the most efficient and reliable way possible. It makes heavy use of the command line as opposed to traditional FTP clients, which greatly speeds up the transfer process (a big deal if you’ve got gigabytes worth of data to transfer).

Just to note, WPEngine has a pretty solid guide here, but they stop short of getting into the technicals, probably for brevity’s sake. It generally boils down to two major steps: copy the wp-content folder from your old site to your new site, and copy the database too.

We’re big fans of WPEngine, a hosting company dedicated to hosting WordPress installations. They do some smart things to keep a WordPress instance speedy that other hosts don’t do out of the box. And just one disclosure: We’re an affiliate of WPEngine, but only because we like them so much.

If you’re thinking of making the switch, sign up through this link, tell us, and we’ll take $10 off your first month of adserving.

Prerequisites

If you’re going to embark on the migration with this guide, you’ll need a few things:

• Familiarity with FTP/SFTP
• Familiarity with the command line (OSX/Linux)
• You old web hosts supports shell accounts (“jail” shell, or jsh is OK). You may have to request this from them.

It really helps to be doing this from a Mac or Linux-based machine too. If you’re migrating from Windows, it may be easier to use the general-purpose WPEngine guide above.

Contents

1. Get the data off your xxisting host
   1. Get the wp-content folder
   2. Get the database data
2. Copy files to your local machine
3. Fix any SQL data
4. Upload SQL data to WPEngine
5. Fix imported WordPress settings rows
6. Transfer wp-content files
7. Tie up loose ends
   1. Make sure site is live
   2. Make sure uploads folder is set correctly
   3. Make sure file permissions are set correctly
   4. Switch domain/DNS settings
8. Wrapping up/Gotchas

1. Get the Data Off Your Existing Host (Highly Technical)

The first part of migrating is liberating your blog data from the old host. The easiest way to make this happen is to get shell access from your host.

Most hosts will enabled limited ssh/shell access to your account by request only. DreamHost will let you enable a user’s shell capability in the FTP accounts section of their admin panel.

You have two tasks here:

• Get the WordPress database data
• Get the wp-content folder

1.1 Getting the wp-content Folder

At this point, you should have shell access to your host. The first thing we need to do is SSH into the host:

ssh username@your-domain.com

Once you’re in, you’ll want to go looking for the wp-content folder, which has all the themes, uploads, plugins, etc. It’s pretty much the content of the site, minus the blog posts and pages. Every host is different, but these are some common locations for the folder:

• ~/www
• ~/public_html
• ~/htdocs
• /var/www/
• /srv/www/

Let’s say the name of our site is broadstreet. We might find our WordPress installation at something like /var/www/broadstreet

You can determine whether you’re at the right place if, inside that folder, you see a bunch of .php files like wp-config.php, index.php, and some folders like wp-admin, wp-content, etc.

Some tools for finding this folder are sometimes available. You can try to run these commands to help you find it:

• locate wp-config.php
• find / -name wp-config.php

Let’s say we found wp-config.php in /var/www/broadstreet. Our next step is to navigate to that folder, if you haven’t already:

cd /var/www/broadstreet

Now we want to package up the wp-content folder and move it to our home directory:

tar -cf wp-content.tar.gz wp-content
mv wp-content.tar.gz ~/

That first command may take a few minutes to complete.

1.2 Getting the Database Content

When the last step is done, it’s time to grab a SQL dump of the database. For this, you need the database host, name, user, password, and encoding/character set. Luckily, all of that is stored in the wp-config.php file. You can view the contents of that file with:

cat wp-config.php

At the top of the file, you’ll see the credentials required to access the database: DB_NAME, DB_USER, DB_PASSWORD, DB_HOST, DB_CHARSET

For the purposes of this example, we’ll say these are our settings:

DB_NAME: broadstreet
DB_USER: user
DB_PASSWORD: password
DB_HOST: localhost
DB_CHARSET: utf8

So, we want to grab our database data in SQL form so we can import it into WPEngine later. We’ll use the mysqldump command to help us with that. Right after, we’ll compress it.

mysqldump -u user -ppassword -h localhost --default-character-set=utf8 -r data.sql broadstreet

Note the lack of a space between the -p flag and actual password. If you add a space, mysqldump won’t be able to log into the database.

That command may take a few minutes to run. When it’s done, you’ll have a file named data.sql that has all of your data ready to go. We’ll want to compress it too, so let’s gzip it and move it to the home directory:

gzip data.sql
mv data.sql.gz ~/

Now we’re done on the server end. We’ll copy those files to our local machine in the next step. Type this to disconnect:

exit

2. Copy the Files You Need to Your Local Machine

Now that those data files are hanging out on their server, we want to bring them down to our personal computers so we can ship them up to WPEngine. Since you have a shell account, you can do this with with the scp utility.

Let’s copy down those files:

scp username@your-domain.com~/wp-content.tar.gz .
scp username@your-domain.com~/data.sql.gz .

Now you’ve got the files you need. As a little bit of prep work, let’s extract the wp-content.tar.gz file and data.sql.gz. On OSX, this is pretty easy. On Windows, you may want to use something like 7zip to help you out.

Once finished, you should have a folder on your computer named wp-content and a file named data.sql.

3. Fix the SQL Data (If Needed)

There’s a chance that your exported database tables had some sort of prefix added to their name. That’s typical for shared hosts that cram a lot of users into a single database instance.

Open data.sql.gz in a text editor like vim, Notepad++, Textmate, etc. You’ll want a file editor that is smart about handling large files, because this one might take a while to open.

Look for a table name prefix. If you see something like this prefixing table names:

INSERT INTO gb3980_wp_users (..., ...)

It’s a hint that gb390_ is a table prefix that was used on your old host. You’ll want to do a find/replace on that file, replacing all instances of gb390_ with nothing, or an “empty string.”

Once that’s done, you’re ready to move on.

4. Upload the SQL to WPEngine

Login into your WPEngine dashboard. At the time of this writing, WPEngine is about to launch a new dashboard, so the steps will be different depending on which interface you’re seeing.

In the old interface, click on the “Admin MySQL” link for your WordPress installation. In the new interface, first click into the “Install” that you’ll be working with. On the left sidebar, you’ll see a link to “phpMyAdmin.” Click that.

You’ll be in phpMyAdmin, which is a well-known web interface for managing a database. Look for the database you want to import into. It will not have the prefix “snapshot_”. For example purposes, We’ll say our DB name is ‘Broadstreet’. There’s a good chance our database is named db_broadstreet.

Click into your database, and you’ll have a list of your tables. Click the “Check All” link below the tables. Then head to the dropdown which says With Selected: and choose “Drop”. Click the “Go” button at the bottom. You may have to confirm the table drop.

Finally, click the import tab at the top. On that page, you’ll see a file selection box. Select the data.sql file that you extracted (and possibly edited in step 3). Pick the character set of the file, which is likely utf8. It’s the same thing from the --default-character-set=utf8 in step 1.

Note that MySQL encodings like latin1 are in that dropdown too, but instead of latin1, it’ll say iso-8859-1.

Click import, and get ready to wait a while. When the page reloads, your data should be imported. If there was a timeout or other error, try again.

If you’re still having trouble, contact WPEngine.

5. Fix Your Newly Imported Database

Now that your data is in the database, we need to fix the “Site URL” entries in the wp_options table. Now that you’re on WPEngine, your URL has changed, and we need to reflect that in the configuration. And keep in mind, we discuss mapping your real domain, like yoursitename.com, to WPEngine later.

Your WPEngine URL is probably something like yoursitename.wpengine.com.

Click the “SQL” tab at the top, and execute this SQL, replacing “yoursitename” with your actual site name:

UPDATE wp_options 
SET   option_value = 'http://yoursitename.wpengine.com'
WHERE option_name  = 'siteurl'
OR    option_name  = 'home'

That should be all. Now you’re ready to transfer your files!

6. Transfer Your Files

The only way to transfer your files to WPEngine is via SFTP. You can set up an SFTP account on the WPEngine dashboard. In the old interface, you click “Manage WordPress”, then click on the “SFTP” tab. In the new interface, you click into your “Install”. Then there’s a panel for managing SFTP logins. Create a login if needed, and keep that username and password handy!

In terms of how you transfer files, there are two ways:

• An FTP client, such as Transmit (OSX) or FileZilla (Windows)
• LFTP — A faster, more efficient FTP client for OSX and Linux

We won’t discuss the details of using a plan vanilla FTP client here. If you don’t know how to use one, you’re probably better off hiring someone who is. But the basic goal is to upload the contents of the wp-content folder you have locally to the wp-content folder you have on your WPEngine server.

Using LFTP can save a lot of time (like, hours-worth) when you have a lot of content to transfer. On OSX, your can easily install it once you have the brew package manager. Once that’s installed, you open a terminal and type:

brew install lftp

On linux, you can run sudo apt-get install lftp or sudo yum install lftp, if it isn’t already installed.

When that’s ready to go, open a terminal and navigate to the wp-content directory. Then, login to WPEngine with lftp:

lftp sftp://yourusername@yoursitename.wpengine.com
cd wp-content

You should have been prompted for a login once you ran the second command. Now, you’re ready to transfer files. Run:

mirror --reverse -c --parallel=4

This will send all of your files up to WPEngine. It may take a couple of hours to complete depending on how much data you’re sending. When it’s done, you’re almost there!

7. Tying Up Loose Ends

At this point, your database has been transferred, and so have your files. Here are some final things to do and check before you call it a day.

7.1 Make Sure Your Site Is Live

When you head to to http://yoursitename.wpengine.com, you should see your site as it once was on your old host, but now on WPEngine. Log into the site by heading to http://yoursitename.wpengine.com/wp-admin.

7.2 Set Your Uploads Folder Correctly

In WordPress, click on Settings -> Media. Make sure the “Store uploads in this folder” field is empty. Some settings from your old host may have been carried over here.

7.3 Make Sure File Permissions Are Set Correctly

File permissions are always a mess once you migrate things via SFTP. There should be a “WPEngine” link in the upper-left part of the WordPress dashboard. Click that. On the page that loads, click the “Reset File Permissions” button. This will ensure that new file uploads will work correctly.

7.4 Make Your Old Domain Point to WPEngine

You probably used to have a website at www.yoursite.com, but now you want that domain to point to WPEngine. This is where you should really know what you’re doing before you make any changes.

There’s no step-by-step here, because everyone’s setup is a little different. But here are some tips for those with different situations:

You Registered the Domain with Your Old Host

If your old site was on something like GoDaddy, midPhase, Bluehost, HostGator, etc, you may have registered your domain with them when you signed up. Since you site won’t be with them any more, you usually have two options here:

1. Keep the domain with your old host and pay for domain hosting only (ask your current host about this)
2. Transfer the domain to a DNS host, like Gandi.net

In case 1, you would eventually end up having to edit the DNS configuration on your existing host, and pointing it to WPEngine’s servers.

In case 2, get an account with Gandi (or some other DNS host). Then, follow this guide: http://wiki.gandi.net/en/domains/transfer

You registered the Domain with Someone Else

This is the less likely, but preferred situation. This happens when you buy domains on a whim, like the author of this guide does, with a registrar account at 1&1, GoDaddy, and many others.

In this case, you should easily be able to find where to edit your DNS settings. Ask your registrar if not (that’s always fun!). Move on to the next step.

What to Do Next

Now, officially, you’ll want to make the www subdomain a CNAME and point it to yoursitename.wpengine.com — and set up the root domain, yoursitename.wpengine.com as a simple redirect to www. Most hosts let you do that.

The reason I say “officially” is that with the method above, you avoid creating A records that point directly to an IP address on WPEngine’s side. This way, if they ever need to reconfigure IPs across their system, you won’t be affected.

If for some reason you really want to create A records and point them to an IP, you can always use the ‘dig’ command in OSX/linux:

dig yoursitename.wpengine.com

That’ll send back the A record value. But don’t go thinking the IPocolypse is and impossibility.

Once the Domain is Reconfigured, Fix the wp_options Table

Remember how we changed the site URL configuration in step 5? Now you need to go back into phpMyAdmin and re-run that SQL, but replace yoursitename.wpengine.com with your domain name (perhaps, yoursitename.com).

8. Wrapping Up

Whew! There were a lot of steps there. In practice, the migration isn’t so long and tedious as it might seem here. We basically wanted to give you a no-gotcha migration guide. That said, here are some nasty things that can cause “gotchas”:

• You come from an instance of WordPress where core files were modified
• Certain plugins may have saved data in the wp_options, user_meta, or post_meta pointing to the old host (filepaths and URLs, in particular)
• Some evil plugins may have saved files (uploads, data) somewhere outside of wp-content

All of those situations are possible, so you should do some sanity checks. If you find yourself battling with one of these, we salute you. There’s some madmen running around in the freelance world doing what they shouldn’t be doing.

If you have any questions, reach out! We realize that with a guide as complex as this, there’s likely to be something we haven’t made completely clear, and may be able to improve.