What you have is a typical iframe hack. there are plenty of postings here on the forums about these.
Typically what may happen is the computer you use to ftp to your website is infected with malware (not all malware is easily detected by your antivirus program). This malware steals your ftp credentials from your ftp program, sending the credentials to the hacker.
Another way is by a security flaw in a 3rd party extension, failure to keep the Joomla core updated, or both.
Even server settings, server software (or lack of), or file/directory permission settings may be the issue.
At any rate of how they got in, you probably found 1 backdoor, here may be others hidden in javascript files or code, hidden in what appears to be photos or images. There are techniques that can really hide the code and the code may change some each time.
I know it is a real hassle, but with somewhere around 3000 files to check line by line, I would really think about removing all your files in public_html Joomla, and any subdirectories (for forums etc.) and replace everything with known good files. Also check the Vulnerable Extensions List Oct - Joomla! Documentation at
http://docs.joomla.org/Vulnerable_Extensions_List_oct to see if any versions of extensions you use are on the list. If it is a later version of one on the list it is probably ok to use, but check with the developer for conformation.