The Joomla! Forum ™

Hi everybody,

There are some extensions (free/commercial) that do this job. However I prefer not to use a lot of extensions so I shall explain how to receive Joomla! database + other databases using simple PHP script and Crontab.

Crontab -for people who are not familar with- is simply a scheduling program that runs a specific commands (scripts) within a scheduled time. You can read about it over Wikipedia.

Here we go for the steps

Step One: Creating backup folder
1- Create a folder on the root of your web server account outside the web folder (public/html) or htdocs, for instance, Backup;

2- Provide it with the famous 777 permission, which I call it the Boing Permission;  :)

Since it is outside the web folder, neither a visitor -even you- nor a hacker can hack into it



Step Two: Creating New User
Depending on your control panel -CPanel in this example- we shall create a new user and apply all Privileges for.

1- from CPanel ==> Advanced ==> MySQL Manager

2 - Under Users; enter User and Password ===> Add User

3 - Under Assign Permissions: make sure to assign this new user for all the databases ==> Submit

Now our dear user is ready
(cont.)

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Step Three: The PHP Script
The internet is full of them. I implemented dbsender from PHPFreaks
http://www.phpfreaks.com/script/view/11.php

1- download ==> unzip/unrar ==> edit using Notepad ++ for example

2 - Go to line 32 and replace the vailues with your own
Make sure to assign the user and pass with the newly created user, and do not forget them




3 - Do not change lines 39, 40, and 41 unless you feel it's good to do so


4 - Line 45 determine the path of the backup script, replace it with the one you've created in step 1

For example: "/home/user/backup"

5 - Lines 47, 48 and 49 determine the e-mail to send the output for


6 - Lines 53 and 54 determine the Subject + Message of the e-mail


you can change what before the variable $senddate + make the body as short as you can


7- Keep the rest of values as they are unless you are an advanced user
(cont.)

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Step Four: Upload the script

1 - Upload to the previously created over step one

2 - give it the permission 711 + make sure the directory is boing (777)

Note: this script deals with one database per file, so in case you need to receive all your databases,
a - copy the file and rename it to another name;
b - edit and change the value of $dbname to the name of the database you want
c- upload ==> 711 file permission
d- repeat for every database.

that's the idea behind having one user for all, so every file has i



Step Five: Run Crontab

1 - From CPanel ==> Crontab ==> Advanced (Unix)

2 - Enter the e-mail to receive from crontab in case of errors. It is not to receive the database backup. differs from t

3 - Enter the command line to run.
Crontab just schedules the script and the command line is as follows

Min  Hour Day Month Weekday Command

Where

• minute (0 - 59)
• hour (0 - 23)
• day of month (1 - 31)
• month (1 - 12)
• day of week (0 - 7) (Sunday=0 or 7)

An Example shall reveal the myth

• suppose a user catswrld;
• the script for joomla database after modifying + uploading is dbsndjm.php
• path is /backup;
• we need to retrieve the backup every Friday at 12:30 pm

So, crontab's command shall be as follows




Another Example

• suppose a user catswrld;
• the script for wordpress database after modifying + uploading is dbsndwp.php
• path is /backup;
• we need to retrieve the backup every Tuesday at 7:15 pm

(cont.)

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Step Six: You've Got Mail
Check your inbox for the backup(s) according to the schedule + scripts.



Note: Scripts

The script I used is just an example, you may surf the internet for better scripts however this is the method you can implement if you wish to use others.

Voila

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Great!  The only issue some users need to realize, is that some e-mail providers have a limit on message size.  So if you have a decent sized DB (over 10 meg), it'll prob exceed that limit.  Just realize this before claiming that it doesn't work!

_________________
Anthony Ferrara - Core Team - Development Coordinator - Bug Squad - JSST

http://moovum.com/ - The Bird is in the air! Get Mollom Anti-Spam on your Joomla! website with Moovur...
http://www.joomlaperformance.com For All Your Joomla Performance Needs

ircmaxell

Thanks a lot for your valuable addition.


In fact, this is why I used this script that deals with one database per php file. There are some other scripts that collect all the databases and flush them into one .tag.gz file which is not good in my opinion.

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

What I did, was created a shell file to create 3 backups (a home directory backup, a /etc backup, and a MySQL backup).  It creates incremental backups every night, then full backups once a week.  Then I wrote a shell script at home to SSH in to the server, download the three files, and write them to a DVD (every night).  So I have daily backups without having to do a thing (since each full backup is around 500 megs, I get about one month on a single DVD).  All I have to do is change DVDs once a month...

_________________
Anthony Ferrara - Core Team - Development Coordinator - Bug Squad - JSST

http://moovum.com/ - The Bird is in the air! Get Mollom Anti-Spam on your Joomla! website with Moovur...
http://www.joomlaperformance.com For All Your Joomla Performance Needs

ircmaxell


This is a very nice and excellent idea

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Here's the shell script to generate the backups (actual shell!)... It places them in 3 directories
/backups/data
/backups/config
/backups/database

_________________
Anthony Ferrara - Core Team - Development Coordinator - Bug Squad - JSST

http://moovum.com/ - The Bird is in the air! Get Mollom Anti-Spam on your Joomla! website with Moovur...
http://www.joomlaperformance.com For All Your Joomla Performance Needs

ircmaxell, Thanks a lot for posting the script  :)

For people who are not familiar with, ircmaxell's script posted in the previous post is a shell script.

I've placed it into a file with .sh extension (attached) and you can run it from the crontab as follows:

1- rename the file to backup.sh not backup.sh.txt as in the attachment;

2 - upload to the root of your domain outside the /public_html directory;

3 Log in to your Control Panel (e.g. CPanel ===> Crontab ===> Advanced Unix style);

4- enter the command to run every


P.S. make replace the values of /home/catswrld/ with your corresponding user values

That's all 

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

HH thanks for the advice, I have tried the script but cannot get it to work, just wondering where I might need to add the database details, i.e. name and password.

Thanks
Lee

Hi lw-d
Lee, kindly check Step Two mentioned previously. Just replace the parameters with yours

I however recommend reading the whole procedure from step 1 if you wish because I've implemented some simple instrcutions to avoid

Edited: OOPS! I'm conflicted. Do you mean my script or ircmaxell's script mentioned on a previous post?

Thanks in advance

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Hi,

Yeah, your script. Will read through again though, in case I missed something.

Thanks

Lee,

The script is working fine with me so just follow up with the steps carefuly + step by step.
In case you missed something again after reading the steps, re-ask

Thanks,
Hicham

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

ircmaxell, Thanks for posting the script and thanks to HH for the zip file and cron command line.

I placed the script in the root of the site  The directory it backs up to is also in the site root (ie. both are outside the public_html directory). I am on a shared server and had to use the path /home/site_username/...etc.  to get things to work as the script below shows.  The only problem I have is after cron runs the script, I get an email saying "tar: Removing leading `/' from member names". Seems that tar would rather have a relative path than the absolute path and throws this error.  Now, I have been unable to work out (limited linux knowlege) how to give tar the proper relative paths so it does not complain and the script will find the directories. So can anyone help with the adjustment of the script to make tar happy?  I know it is probably a rather simple fix.

Thanks,
Phil

_________________
Phil D
Unrequested private messages and/or emails may not get a response.
Please give me a simple question, so that I don't have to think, because if I think, I might find answers that don't fit the question.
http://forum.joomla.org/ Security Moderator

That's normal... You don't need to worry about that...

_________________
Anthony Ferrara - Core Team - Development Coordinator - Bug Squad - JSST

http://moovum.com/ - The Bird is in the air! Get Mollom Anti-Spam on your Joomla! website with Moovur...
http://www.joomlaperformance.com For All Your Joomla Performance Needs

I guess I wasn't all that worried about it as the script is working.  Just was bugging me and I wanted to see if I could eliminate it.

Thanks,

Phil

_________________
Phil D
Unrequested private messages and/or emails may not get a response.
Please give me a simple question, so that I don't have to think, because if I think, I might find answers that don't fit the question.
http://forum.joomla.org/ Security Moderator

Well, just in the crontab, just redirect the output
backup.sh 1>/dev/null

_________________
Anthony Ferrara - Core Team - Development Coordinator - Bug Squad - JSST

http://moovum.com/ - The Bird is in the air! Get Mollom Anti-Spam on your Joomla! website with Moovur...
http://www.joomlaperformance.com For All Your Joomla Performance Needs

PhilD



Crontab can send you an e-mail In case any errors occurred during the process of running the script via this /dev/null, so make sure that you've placed your e-mail within Crontab right.

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

HH,

Yea, I am currently using this in the crontab:

/home/site_username/backup.sh > /dev/null

The cron is what is sending the email with the tar complaint tar generates about absolute paths ( tar: Removing leading `/' from member names ).  I believe cron will still send errors to the email address given even with the > /dev/null on the end.  It just won't send a log of what all it performed when running the job.  On the plus side I know that the script runs everyday.  It might even be fun get a few emails with all the output from cron when the script is run.

------------------------- How I am getting the files to my computer. ----------------------------

Right now I am using a simple .bat file I wrote under win xp on a schedule to ftp to the website and copy the backup files off the website.  It works, but it is not very smart.  It just gets everything in the folder everyday with extensions of *.gz *.tgz 

The problem with that is it gets everything from the day(s) before again and again if run daily.  It would be nice if it would only get the current days backup files and not ones in the past again and again.  I could have it delete the files from the server after it downloads them but I think the crontab script will already remove the backups once a week on sunday if I'm reading the cron script right. 

I think it would be better to find a download script that can figure out what backup files (*.gz *.tgz  ) are current on the server and get only those files everyday. Something that would write to and keep a weekly log of filenames,  compare the filenames of the files on the server to the filenames in the log and only download the *.gz *.tgz  ones that didn't match. .

Another way could be to just get the current date and compare with the date in the file name and then download only those files that match.

Or I could just run the download script once a week before the crontab is run so it can download the weeks files.  It's not as safe though as your leaving your data on the server so it negates the purpose of the differential backups in the event of a server crash.

Anyway if anyone has any thoughts on this I'll be happy to hear them.

PhillD

_________________
Phil D
Unrequested private messages and/or emails may not get a response.
Please give me a simple question, so that I don't have to think, because if I think, I might find answers that don't fit the question.
http://forum.joomla.org/ Security Moderator

I do exactly what you wrote and receive mail from server but file is empty?

yellowtoot
Sorry for the 'somehow' late reply.

1. Are you sure that you created + assigned a new database user as explained in "Step Two"?
2. Are you sure that the scripts are uploaded to the same directory where you want backups to be placed?

p.s. A basic question/wonder: is the problem solved during the period since you posted and I replied?

Thanks

_________________
Me = Wonder + Ponder
http://www.hichamaged.net/

Does this script backup.sh also need 711 permissions?

It will need 711 permissions to work. On some srever setups (php as cgi) you can use 700.

_________________
Phil D
Unrequested private messages and/or emails may not get a response.
Please give me a simple question, so that I don't have to think, because if I think, I might find answers that don't fit the question.
http://forum.joomla.org/ Security Moderator

Thanks Phil. I have set the script to 711 and it at least seemed to have ran the script. Last night I only got a one-line email result with "permission denied".

Tonight, this was in the email notice:



This is my script:



I already have /backups created, but I'll try creating the respective directories /backups/data, config, and database with 777 permissions... but how do I get around the "/usr/bin/find: Permission denied" error?

Thanks in advance.

You will have to ask your hosting services tech support if they allow the find command and/ or if they have changed the find command to something different.

Many hosts will change the find command and certain other commands to something else for additional security. I know MediaLayer does this as I had to ask for the proper find command.

Some hosts also may not allow the find command to be run, and you will have to ask for their suggested alternative.

_________________
Phil D
Unrequested private messages and/or emails may not get a response.
Please give me a simple question, so that I don't have to think, because if I think, I might find answers that don't fit the question.
http://forum.joomla.org/ Security Moderator

Found someone with a similar problem on the SG forums:



Does anyone have alternative suggestions to the find command or should I try to contact support?

The quote is correct and is what was alluded to in my last post.

Sometimes rather than spinning your wheels for days looking for an answer, it is better to just contact support, tell them what your doing (attach a copy of the script to the support ticket) and ask them what the find command is on their servers or what alternative command they suggest. Many hosts have changed the find command to something like "xyzfind" for security reasons. You have to ask support to get the proper command syntax.

Some hosts just do not allow "find" in any form and support can also tell you this. If this is the case, you will not be able to use any script that uses "find" on that hosts servers. I would ask support how they expect customers to do their own automated backups of a site and databases. If you don't like their answer, I would consider switching hosts.

_________________
Phil D
Unrequested private messages and/or emails may not get a response.
Please give me a simple question, so that I don't have to think, because if I think, I might find answers that don't fit the question.
http://forum.joomla.org/ Security Moderator

Thanks for your explanation Phil. I will contact the webhost and in hopes that my answer will benefit someone else in the community, will post my results.

Cheers

Unfortunately, the host (Siteground) has refused to help. They want to charge $ for creation of a "sophisticated" daily backup script. They said to "restructure the backup script and make it create a directory and store all files for a given time frame into that directory."

Does anyone have any suggestions on an alternative .sh that does not use find?