Mauserrifle.nl Tech and Life

    Home     Archive     Projects     Contact

Checklist Website Migration on Linux Web Server with Apache2

This article covers the checklist I use when migrating a website on a Linux web server. It aims at a downtime of 1-5 minutes.

It covers the following tech:

  • Debian
  • Apache2
  • PHP fpm
  • MySQL

Most commands in these instructions will work fine on other Linux distributions.

Use this checklist as a base for your own specific configurations. Nginx won’t be much different.

This checklist is quite long because it is aimed to first time Linux migration users or a single website. Eventually when you are familiar with all the steps, it will be easier to make scripts for your migrations. Scripts are very useful with incremental migrations that have to be run multiple times (for instance when building a new version of a website).

It is always good to come back and review a checklist such as this.

Note: I will cover incremental example scripts later in another post.

General tips

Check user. Always run version control (git) and build processes (yarn/npm) as the default user from a folder (check with ls -l. Use the root user only where necessary.

Prevent VIM code format. When you copy/paste configs via your own terminal from server A to server B, always put vim in paste mode and then insert mode:

  • Open vim
  • Type using command mode: : set paste
  • Activate insert mode: I
  • Paste into your terminal

General preparations

  • Check the DNS A record TTL of the affected domains using the command: dig A website.nl. Lower to 300 seconds if required.
    • Can’t change to less than a day? And the website is static? No worries.

    • Can’t change to less than a day? The website is dynamic? And you you don’t want people to see a maintenance page for a day? Then consider a proxy on the old server.

      Two servers running the same dynamic website will cause conflicting data (such as webshop orders).

      An example for apache2:

      #sudo a2enmod proxy
#sudo a2enmod proxy_http

 <VirtualHost *:80>
   ServerName website.nl
   ServerAlias www.website.nl

   ProxyPreserveHost On
   ProxyPass / http://IPADDRESS:80/
   ProxyPassReverse / http://IPADDRESS:80/
</VirtualHost>
  • Check the SPF record using the command: dig TXT website.nl. Check whether “a” is in it. Add if missing.
  • Notify customer about migration work.

Preparing new server

  • Create domain folders 
    $ mkdir -p /home/domains/website.nl/htdocs /home/domains/website.nl/www-log

  • Copy old server FPM pool or create a new one.

    Example file:

    /etc/php/7.3/fpm/pool.d/website.nl.conf

    [website.nl]
listen = /var/run/website.nl.sock
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
user = www-data
group = www-data
chdir = /
pm = ondemand
pm.max_children = 5
pm.process_idle_timeout = 10s
pm.max_requests = 200
php_admin_value[open_basedir] = "/home/domains/website.nl:/tmp"
    • Restart FPM: systemctl restart php7.3-fpm
  • Create new vhosts in /etc/apache2/sites-available/. Inherit from old server. (website.nl.conf and website.nl-le-ssl.conf).
    • Do not migrate letsencrypt certificates. Remove them from the vhost. Request them again after going live. Other purchased certificates can be transferred though.

    • Add FPM socket to the vhost

      <FilesMatch \.php$>
  SetHandler "proxy:unix:/var/run/website.nl.sock|fcgi://localhost/"
</FilesMatch>
  • Enable vhosts. Debian example (based on conf name):
    • a2ensite website.nl
    • a2ensite website.nl-le-ssl
  • Run Apache check apachectl configtest
    • Consider running the better apache2 config test /root/apache.check.sh that also checks missing www-log folders (which prevents apache2 from starting).
  • Restart Apache2
    • systemctl restart apache2
  • Create database and user with the same credentials (using cli/phpmyadmin).
    • See website config on old server (e.g. .env, wp-config.php)

Data migration

This takes a while the first time, but will take seconds to complete the next time. We run this as root so we don’t have permission issues and all ownership of files are properly set.

  • New server
    • Temporarily allow root login in /etc/ssh/sshd_config (or use preferred NOPASSWD alternative below)
      • Add/Enable: PermitRootLogin yes
      • Comment AllowUsers if set (add #): #AllowUsers USER
      • Restart SSH: systemctl restart ssh
    • Preferred NOPASSWD alternative: Allow sudo rsync commands without password.
      • Run visudo. Add: YOURUSER ALL = NOPASSWD:/usr/bin/rsync
  • Old server

    • Run rsync

      Root method:

      sudo rsync -az --include=".*" --modify-window=2 /home/domains/website.nl/ root@NEWSERVER:/homeains/website.nl/

      NOPASSWD method:

      sudo rsync -az --include=".*" --rsync-path="sudo rsync" --modify-window=2 /home/domains/website.nl/ USER@NEWSERVER:/homeains/website.nl/

Database migration

Warning: Never put sql files in public website folders (such as htdocs).

  • Old server

    • Export database

      mysqldump --user USER --skip-add-drop-table -p DATABASE_NAME > /tmp/DATABASE_NAME.sql

    • Transfer to new server

      rsync -a /tmp/DATABASE_NAAM.sql USER@NEWSERVER:/home/domains/website.nl/
  • New server

    • Import database

      mysql -u USER -p DATABASE_NAME < /home/domains/website.nl/DATABASE_NAME.sql

Testing on new server

  • Adjust your own host file with NEWIPNUMBER website.nl www.website.nl so that you can test
  • Possibly make small changes in PHP on the new server (backwards compatible)
    • commit and push git
    • Also pull changes on the old server. Because when going live we do a new data migration (rsync)

  • Check whether the vhost is really okay and letsencrypt can access files later.

    echo "test" > /home/domains/website.nl/htdocs/.well-known/acme-challenge/test

    Access with the browser: http://website.nl/.well-known/acme-challenge/test

Other vhosts checks on old server

  • Check whether the website name is also used in other vhosts (e.g. redirect only vhosts) and migrate them as well
    • Search: grep -i 'website name' /etc/apache2/sites-enabled/*
    • Check to be sure if the A record of the ServerName and ServerAlias ends up on the old server at all (check: dig A website.nl). If not, no migration is required.

Everything OK?

  • Notify customer about migration date

Going live

  • Put the old server site on maintenance. Include a 503 header.

    Example PHP for adding at the top of index.php or a config file:

    header('HTTP/1.1 503 Service Temporarily Unavailable');
header('Status: 503 Service Temporarily Unavailable');
die('This website is currently under maintenance.<br>Please try again later.');
  • Rename old server db user (eg * _bak) so that there are no more mutations
  • Disable old server cronjobs by commenting them out
    • Usually in /etc/cron.d/website
    • Search to make sure: grep -i 'website name' /etc/cron.d/*
    • Also check crontab -e for different users
  • Perform data migration again
    • Double check: PHP fixes also pulled on old server?
  • Perform database migration again
    • Delete test db on new server first (via cli/phpmyadmin). NOTE: Double check which server you are on before removing!
  • Double check that everything really works
  • Change the A (ip4) and AAAA (ip6) DNS records of all moved domains. Check ServerName and ServerAlias.
    • Reminder: Make sure the domain A record was set to the old server before changing it (check: dig A website.nl). You checked this earlier in this checklist.
  • Fix Letsencrypt SSL
    • Run: certbot. Select website.nl and www.website.nl
  • Transferring Cronjobs
    • Usually in /etc/cron.d/website
    • Search to make sure: grep -i 'website name' /etc/cron.d/*
    • Also check crontab -e for different users
    • Remove any comments added before
  • Remove maintenance page code on new server so that visitors can access it
  • Disable temporary root login or NOPASSWD on the new server if this was the last migration

Aftermath domain

  • Schedule deleting the website on the old server (wait 1 month). If the entire server is eventually canceled, you can skip this step.

Aftermath server

  • When canceling the entire old server, secure the backups!

    The rsync command below transfers hard links too (important for incremental backups).

    sudo rsync -az -H --include=".*"  --modify-window=2 /home/backup/rsnapshot DESTINATION

Template data transfer

When you are familiar with all the steps above. It is easier to just make scripts for your migrations.

The template below can be used for db dump and rsync. It is a simple version of many steps above. It uses a single rsync command because the mysqldump happens before the data migration.

RUN ON OLD SERVER

Export db and rsync it with all website files to the new server.

mysqldump --user USER --skip-add-drop-table -p DB_NAME  > /home/domains/website.nl/DB_NAME.sql
sudo rsync -az  --include=".*" --modify-window=2 /home/domains/website.nl/root@NEWSERVER:/home/domains/website.nl/

Consider adding --exclude=".env" --exclude="node_modules" on incremental runs.

When using NOPASSWD add: --rsync-path="sudo rsync"

RUN ON NEW SERVER

Drop the last test db and recreate it.

mysql -u USER -p -e "drop database DB_NAME; create database DB_NAME;"
mysql -u USER -p DB_NAME < /home/domains/website.nl/DB_NAME.sql


I will post real life incremental migration examples soon. Scripts I used for months during development of a new website.

Thanks for reading!

If you liked this post, you can share it with your followers!