Large number of images slowing down content edits

Discussion regarding Joomla! Performance issues.

Moderator: General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Security and Performance FAQs
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Locked
ddmobley
Joomla! Intern
Joomla! Intern
Posts: 90
Joined: Thu Jun 15, 2006 2:18 am

Large number of images slowing down content edits

Post by ddmobley » Fri Jan 15, 2010 2:06 am

I have been running a fairly successful Joomla website since the summer of '06, accumulating a little over 3000 articles. The site is v1.0.15 and my image strategy was never very concrete, but I knew I needed some way to make finding images easier when creating or editing articles.

What I did was create a directory under "images/stories" called "content" and then placed each article's specific images in a folder under "content" that was named the same as the article id. In other words, if you were viewing an article with an id of 852, then the images for that article would always be found in "images/stories/content/852". I had a few other folders under "images/stories" for images that were used often.

I now have a little over 7500 images in the "images/stories" directory tree, and I am noticing a longer and longer delay when trying to start a new article and editing and existing article. I know this is because of the humongous number of image being shoved into the folderimages[] array. The page source alone for one of the articles being edited is 777K (I know, I viewed source, gasped at the 7500+ images and then saved the file to see how big it was).

Does anyone have any recommendations on how to resolve this issue? Please don't say update to J1.5, while that is going to happen, it will be some time down the road. I am not sure 1.5 would handle this situation any differently anyway.

I have Gzip on, added "php_flag zlib.output_compression on" to .htaccess (that really sped the front end up, didn't much affect the backend). I searched Google for some time, wondering if anyone had modified the "MosImage Control" to handle images differently but came up empty handed. I guess the ideal situation would be to build an array of folder, but don't fetch the image names in a particular folder until it was clicked on. As it is, I am loading 3,000+ folders that I am not going to even use and the performance hit is starting to be very evident.

Anyone familiar with this?

ddmobley
Joomla! Intern
Joomla! Intern
Posts: 90
Joined: Thu Jun 15, 2006 2:18 am

Re: Large number of images slowing down content edits

Post by ddmobley » Thu Jan 21, 2010 7:37 pm

Do you mean no one has a Joomla website with over 7,000 images in the stories directory who has encountered this problem? Certainly there is someone out there who has some input on this...

theslacker
Joomla! Apprentice
Joomla! Apprentice
Posts: 10
Joined: Mon Jan 25, 2010 8:55 pm

Re: Large number of images slowing down content edits

Post by theslacker » Mon Jan 25, 2010 9:08 pm

I'm having the same problem and have been investigating this. Back in 2006 someone posted they had created an AJAX solution to handle this. I think we are going to need to write our own AJAX solution (or get theirs).

I have started to fiddle with this. Inside of /administrator/components/com_content/admin.content.php, I added in an input box with javascript, so I can enter the directory path of an image folder. I have it so when you enter the path and click off the box, it will load the files that are in that folder in the left hand box.

What needs to be done, is when the path changes in my little input box, an AJAX call needs to be made to the server to just grab the files in that folder. Then that list needs to be send back and loaded on the page.

If anyone can whip this together easily, it would be REALLY handy. Otherwise at some point I'll have to try to tackle it myself. :(

What happens with the stock joomla code, is that when you click on a content article to edit it for example, joomla is traversing the entire images directory recursively, building the array, and then dumping the entire thing in the output to the browser. This isn't a very scalable design if you have a section/category that is really large.

ddmobley
Joomla! Intern
Joomla! Intern
Posts: 90
Joined: Thu Jun 15, 2006 2:18 am

Re: Large number of images slowing down content edits

Post by ddmobley » Tue Jan 26, 2010 12:47 am

Funny that you should respond to this today, because I just finished up implementing my solution to this situation. I did it by rewriting Joomla's implementation of the image management for Sections/Categories.

Joomla allows you to specify the directories to corral the article creation/editing function into. It defaults to the global images in the "images/stories" directory. In its default arrangement, you have access to any images and directories you create after the site is running.

For example, if you were using the default configuration, and you added a new directory "news" under "images/stories", any images you put there would be visible to you in the Mos Image browser. For most people, that works, but like you have discovered, when you get a lot of images in that directory structure, it burdens the browser down to the point of putting you to sleep.

Now let's say we have directories for "news", "reviews", and "support" articles and you go into the Section or Category Manager and assign each section or category to only use/display images from the respective directory for each section/category. You are good. Until you add a subdirectory.

I segregate each article's supporting images into their own directory, so article #2982 would have all of its photos/videos/etc. in a directory named "2982" under the "images/stories" directory. If it was a news article, by definition, the directory path should appear as "images/stories/news/2982/". If I had defined definitive, specific directories in the Section/Category Manager, such as "news" for the News section or category, the Mos Image browser could see anything in "images/stories/news" but it could not see the new directory "2982" I had just uploaded. This was unacceptable to me.

So, I modified the Joomla Content Editor to allow the Mos Image browser to traverse the directories specified in the Category or Section Manager and add the files in the directories beneath the top level to the image list on page load. This would allow me to define a top level directory or directories in the Section or Category Manager for each entry and let the Content Editor sort it out at run time. Works really well, and I have restored the page load speeds back to what they used to be before it had to sort 7000+ images. Now, the page sorts about 1500 for a given section.

The modification required inclusion of the function "recursive_listdir()" from the Category Manager into the Content Manager and then the coding into the portion of the function "editItem" where it builds the folder list. About 15 lines of code plus the new function.

theslacker
Joomla! Apprentice
Joomla! Apprentice
Posts: 10
Joined: Mon Jan 25, 2010 8:55 pm

Re: Large number of images slowing down content edits

Post by theslacker » Tue Jan 26, 2010 2:41 pm

I'm happy to hear you coded something that works well for you.

For me, unfortunately that wont work. Almost all of my content is under one section/category (its the part of the site we write the most content for). So, even if I just had Joomla look at it's folders, it still would be the bulk of images that need to be traversed.

I think the only way to completely future proof this is to do the AJAX type of thing (or something like that), that just reads the files as you need to. That will make this thing work even if there were 100,000 images in one section (as long as they are spread way out over sub folders like you and I do).

I don't have much experience with php and ajax, but I think I'm going to try to get something working.

theslacker
Joomla! Apprentice
Joomla! Apprentice
Posts: 10
Joined: Mon Jan 25, 2010 8:55 pm

Re: Large number of images slowing down content edits

Post by theslacker » Wed Jan 27, 2010 8:31 pm

It took me about 8-10 hours, but I wrote the AJAX solution. It works awesome. Load time went from 1 minute to 3 seconds. My implementation is a little crude, but for me it will do what is needed. Now I can have unlimited sub folders with no silly slow down.

ddmobley
Joomla! Intern
Joomla! Intern
Posts: 90
Joined: Thu Jun 15, 2006 2:18 am

Re: Large number of images slowing down content edits

Post by ddmobley » Wed Jan 27, 2010 8:43 pm

I would be interested in previewing your work if I could. I may be able to enhance it a bit if you want more functionality. Let me know if we can do this.

theslacker
Joomla! Apprentice
Joomla! Apprentice
Posts: 10
Joined: Mon Jan 25, 2010 8:55 pm

Re: Large number of images slowing down content edits

Post by theslacker » Wed Jan 27, 2010 8:53 pm

I'd like to PM you, but need 5 posts on this forum.... almost there.... :) pm me.

ddmobley
Joomla! Intern
Joomla! Intern
Posts: 90
Joined: Thu Jun 15, 2006 2:18 am

Re: Large number of images slowing down content edits

Post by ddmobley » Wed Jan 27, 2010 9:34 pm

Maybe I can help you! :D

theslacker
Joomla! Apprentice
Joomla! Apprentice
Posts: 10
Joined: Mon Jan 25, 2010 8:55 pm

Re: Large number of images slowing down content edits

Post by theslacker » Wed Jan 27, 2010 9:38 pm

one more post so i can pm


Locked

Return to “Performance - 1.0.x”