Replicating base server -> target server (Digital Ocean/Fed/Centos)
Hi.
Kind of long ,but might be useful/helpful to others.. Feel free to
comment as you see fit!
I'm using this as a step/guide to recreate/replicate a smaller
droplet/vm from a larger vm within Digital Ocean.
The goal:
Replicate/reproduce the users/processes/directories/files
from a base server to a target server.
To be able to then use the target server in place of the base/initial
server
The base server acts as a nfsClient server to a nfsServer
Doing this to create a smaller vm/droplet from Digital Ocean (DO), and need
to "replicate" a larger droplet/vm.
DO doesn't provide a way to accomplish this!
::::---
This process doesn't deal/handle any iptable/firewall/selinux/dns
issues....
-need to figure out how to deal with these....
Process:
-analyse the base server to get all the existing users
-analyse the base server to get all the existing/running system processes
-analyse the base server to get all the existing/running 3rd party processes
-analyse the base server to get all the existing dirs/files
-analyse the base server to get all the existing sshkey data/files
-analyse the base server to get all the installed rpm/packages
-create process on the target to generate the user/group/passwd
for the users on the base server
-create process on the target to generate the ssh key for the users
to replicate the base users/sshkeys
-create process to copy all the dir/files from the base to the target,
excluding a limited subset of dirs..
-create process to install on the target, all the installed packages from
the base
-
Steps::
-on the target disable selinux for simplicity
vi /etc/sysconfig/selinux
set >> SELINUX=disabled <<
-Create the list of users
-On the target, generate the users/passwd/groups. It appears this
could/should be doable by copying the requisite files from the
base->target with the associated perms..
Decided to do it manually to ensure it matches..
-the users are/were:
root/root_tmp/test_user
-use useradd/passwd to generate the same users/passwd/group as well as
the same uid/gid for the userID/groupID to match the base server
--match the user/passwd/uid/gid so the target matches the base
--if required, mod the uid/gid
usermod -u xx test_user
groupmod -g xx test_user
--at the same time, change/match across the entire dir for any files
to handle user/group owner (do this for all users)
find / -group 500 -exec chgrp -h test_user {} \;
find / -user 500 -exec chown -h test_user {} \;
-change to given user (root/root_tmp/test_user)
set up the ssh keymkdir ~/.ssh
chmod 700 ~/.ssh
echo '' > ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
chmod 755 ~ ~/.ssh
update/cpy in the "authorized_keys" file the pub key from the
base/user "authorized_keys" file
at this point, the users are set, and the ssh keys are set..
----------------------------------
Copy the dirs/files....
The process runs on the base, copying/rsync from the base-> target handling
the entire disk from the "/" top.. on down..
The excluded list follows as well as as the rsync cmd..
The file "/etc/skipdirs.rsync" contains:
/proc/*
/sys/*
/dev/*
/media/*
/var/log/*
/var/log/journal/*
**/.cache/google-chrome/***
**/.ccache/***
/BACKUPS/*
/run/media/*
/var/lib/nfs/*
/usr/src/kernels/*
/root/.cache/*
/swapfile
/bin/*
rsync --progress -avAI --exclude-from=/etc/skipdirs.rsync / root@11.22.33.44:/
(shout out to Rick!!)
I intentionally wanted to see all the files xfered as I ran/run the cmd..
-the cmd is run as "root" on the base, to ensure the process has
complete access to all dirs/files..
-the "root" user on the target has access to top level as well..
RPM packages..
-To further ensure the target will mtach the base,
on the base as "root" run
rpm -qa > rpmlist.dat
rsync --progress -avAI /rpmlist.dat root@11.22.33.44:/
and on the target as root
cat /rpmlist.dat | xargs yum -y install
followed by
yum update
--end result is that all the packages on the target should match
the base
::given that this is going to be a "client" to the nfsServer..
-make sure the nfs client utils are available.. they should already
be but check anyway..
on the target as root
install any/all additional packages on the target::
yum install nfs-utils nfs-utils-lib
as root
sudo cat << EOF > /etc/yum.repos.d/google-chrome.repo
[google-chrome]
name=google-chrome - \$basearch
baseurl=http://dl.google.com/linux/chrome/rpm/stable/\$basearch
enabled=1
gpgcheck=1
gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub
EOF
yum install -y gcc xorg-x11-server-Xvfb google-chrome-stable
yum install xvfb x11-xkb-utils
yum install xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic
yum install vi
pip install pyvirtualdisplay
python -m pip install --upgrade pip setuptools wheel
easy_install simplejson
yum install libxml2-python
yum install epel-release
pip install -U selenium
pip install pyvirtualdisplay
yum install Xvfb libXfont Xorg
-finally.. install the "stuff" for the selenium/firefox/chrome testing..
wget
https://github.com/mozilla/geckodriver/releases/download/v0.19.1/geckodri...
tar -x geckodriver -zf geckodriver-v0.19.1-linux64.tar.gz -O >
/usr/local/bin/geckodriver
chmod +x /usr/local/bin/geckodriver
rm geckodriver-v0.19.1-linux64.tar.gz
ln -s /path/to/file /path/to/symlink
ln -s /usr/local/bin/geckodriver /usr/bin/geckodriver
## Chromedriver
wget https://chromedriver.storage.googleapis.com/2.35/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
sudo chmod +x chromedriver
sudo mv chromedriver /usr/local/bin/
rm chromedriver_linux64.zip
ln -s /usr/local/bin/chromedriver /usr/bin/chromedriver
finally.... on the base....
as root..
vi /etc/ssh/sshd_config
#PermitRootLogin yes
PermitRootLogin no
ok....
if you can see something else that can be done that I might have
missed.. or that would be better.. shout it out.
At some point.. others on DO will be looking for something like this.
Ideally, it would be cool/good to have this in a kind of chef/puppet cookbook.
But that's above my pay grade for now!
thanks