How to Merge Multiple Apache Log Files when Migrating Servers.

Here’s a little howto on merging multiple apache log files when you are migrating servers or want to consolidate your logs into one place. This works on ubuntu and debian, but your mileage may vary on other OS’s.

Here’s the howto:

Login to old server
scp old log files over, to the admin home directory, in /logs
```
 # scp -r /home/username/logs/ <username@yournewserver>:
```
If copying multiple directories, and you want to call them something other than logs do this:
```
 # scp -r /home/username/logs/ <username@yournewserver:/home/username/logs_domainname/> 
```
Login to new server
Go to /home/username/logs (or /home/username/logs_domainname)

Make a backup of current logs

 #mkdir new_logs  
 #cp access_log error_log ./new_logs

Make a few directories

 # mkdir old_logs  
 # mkdir concat_logs

Enter the old logs directory, and copy the old log files over
```
 # cd old_logs  
 # cp /home/admin/logs/* .
```
Now we’re going to concatenate all these to a new single file. Logresolvemerge.pl is part of the awstats distribution and has been copied to /usr/local/bin.
```
 # logresolvemerge.pl ./access_log* > access_log_overall_old  
 # logresolvemerge.pl ./error_log* > error_log_overall_old
```
Now we’ll leave this directory, and enter the concat_logs directory
```
# cd ..  
# cd concat_logs
```

Now we’ll copy our newly created log files over to this directory

# cp ../old_logs/error_log_overall_old ../old_logs/access_log_overall_old .

Now we’ll copy in the “new” log files from the new production server. This step can be omitted if there are no stats on this server already, or they are not needed.

# cp ../new_logs/* .  
# ls -l

-rw-r–r– 1 root root 554 Sep 2 19:38 access_log  
-rw-r–r– 1 root root 3702368 Sep 2 19:37 access_log_overall_old  
-rw-r–r– 1 root root 115 Sep 2 19:38 error_log  
-rw-r–r– 1 root root 907085 Sep 2 19:37 error_log_overall_old

Here we’ll combine the old logfile, with the new logfile. This again, can be omitted if not using old stats and new stats.

# logresolvemerge.pl ./access_log ./access_log_overall_old > access_log_final  
# logresolvemerge.pl ./error_log ./error_log_overall_old > error_log_final

Now we’ll either have in our directory, a final old log file, or a final combined file from this server, and the old server up to the minute.
```
   # cp ./access_log_overall_old ../access_log  
   # cp ./error_log_overall_old ../error_log  
   # /etc/init.d/apache restart
```

If you did do step’s 12 and 13, then do this:

# cp ./access_log_final ../access_log  
# cp ./error_log__final ../error_log  
# /etc/init.d/apache restart

You can now delete the old_logs, new_logs, concat_logs directories if wanted.

# cd /home/domain/logs/  
# rm -rf concat_logs  
# rm -rf new_logs  
# rm -rf old_logs

To update the logs, and make them visible on the awstats interface run this:
```
# /usr/lib/cgi-bin/awstats.pl -config=yourdomain.tld -update wq
```
Visit them on the web at; http://www.domainname.com/awstats/awstats.pl
Add the update line to your crontab to schedule it. You can also use updateall.pl that comes with the awstats distribution to update all sites at one time if vhosting.
Go have a rum and coke to celebrate!