Well...I would say that depends.

Can you find out what user Apache is running as? RobS is right about permissions, those commands are invaluable. But you would still have problems on directories that Joomla! like to write to, if you owner for FTP is different that Apache. Basically this is the best way:
1. Use RobS example and get all your permissions the same. For this to work, in a perfect world, your Apache user for your VirtualHost will be the same as your FTP user, therefore the owner is the same whether you install or upload a file through Joomla! or through FTP. This is known as SUExec, a very handy utility in *NIX boxes for changing the user or a process, in this case Apache. If this is the case, and your host will be able to tell you whether or not it is, then yes you will be fine with only 755, 644; this is the preferred method as well.
Unfortunately, when someone is controlling the server, using SUExec isn't always the case. If this is the situation, here are your two options:
1. Use a test server and only modify files locally, then upload changes through FTP. This way you keep your FTP user for all files and folders, except of course for files & folders your users need to be able to change, i.e. for picture uploading or document sharing. This way you can use the FTP owner for all files and keep the 755, 644 permissions for all folders except for those which Apache needs to make changes to, i.e. your cache folder or your images folder or something like that. Because you will use your test server to install extentions and such, Apache doesn't need rights to write to these, only to read them. For those folders that need world writeable, just use 777. That's not the best solution, but it works.
2. This is what I used to do when I was on shared hosting: Keep the Apache user as the owner of all files, and maintain 755, 644 permissions for all resources. When I made changes from on my test server, I would create an incremental tar archive of the changes and upload with FTP. At this point the tar archive is owned by my FTP user, but using Apache I would unzip & untar the archive and overwrite and all files on the live site. This essentially patched my live server with my changes and because Apache was the one doing the overwriting, the files were still owned by apache, thus 755, 644 were still valid.
All three solutions will work, I have used them all. Like I said, if your FTP and Apache user are the same, that is great, makes things much easier, and a little safer in my opinion. The second option would be to do what most people do, just upload with slow old FTP your changes, keep your FTP user owner for all permissions. But keep in mind you will need 777 for any folder you want Apache to write to. So if you don't have a test server, then the recommend folders will all have to be 777, not a good options if you ask me, but there for compatibility. Finally you could make Apache own all files and manage your server from the live site or a test server using tar archives to apply patches. There are two problems with this method though. First, if you server is not hardened properly, it is possible for others to use the Apache user to access and modify your files. This is anytime you are on shared hosting and multiple users share a common account, like Apache. Secondly, in my case, my genius host thought every few weeks they would chown -R all my resources back to my FTP user totally jacking up my users as they could no longer write to the images directory for photo uploads.
That's my 2.5 cents. Good Luck.

Jeff McCoy