Smart Search index never completes, mem limit at 400MB Topic is solved

Need help with the Administration of your Joomla! 4.x site? This is the spot for you.

Moderator: General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Windows Defender SmartScreen Issues <-- please read this if using Windows 10
Post Reply
maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Smart Search index never completes, mem limit at 400MB

Post by maestroc » Tue Nov 15, 2022 1:50 am

I have been unable to get my Smart Search component to fully index my site. It seemed to pull in around 100 or so items the first time but now will go no farther. When I run it again it advances to the point shown below but goes no further always stopping at 7 out of 490. If I let it set there eventually the box goes away but the list of indexed items is missing tons of articles. Even the old style J3 search module returns dozens more hits than the smart search does. Am I missing something?

Indexer Running
Your content is being indexed. Do not close this window.

Tags
2 of 2
Categories
14 of 14
Contacts
2 of 2
Content
7 of 490
Newsfeeds
0 of 0



I am running 4.2.4 on PHP 7.4.33 with a 400MB memory limit in PHP (confirmed by system info screen).

User avatar
toivo
Joomla! Master
Joomla! Master
Posts: 15484
Joined: Thu Feb 15, 2007 5:48 am
Location: Sydney, Australia

Re: Smart Search index never completes, mem limit at 400MB

Post by toivo » Tue Nov 15, 2022 5:43 am

Does the PHP error log have any warnings, notices or actual errors, triggered by the indexing? Change the level of error reporting in Global Configuration to 'Maximum' and turn on the Debug option so that possible messages are displayed with some detail about their origin.

More information required. Some third party extensions installed at the website may have Finder or Content plugins that interfere with indexing.

Please post the results from the Forum Post Assistant (FPA) by following the instructions at https://forumpostassistant.github.io/docs/ so that our volunteer experts can review the configuration.
Toivo Talikka, Global Moderator

User avatar
ceford
Joomla! Ace
Joomla! Ace
Posts: 1496
Joined: Mon Feb 24, 2014 10:38 pm
Location: Edinburgh, Scotland
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by ceford » Tue Nov 15, 2022 10:57 am


maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by maestroc » Sat Nov 19, 2022 1:37 am

FPA info below.
Nothing showing in the error log even with reporting on maximum.

Also getting occasional of out of memory errors in the back end. Never see them on the front end though.

Symfony\Component\ErrorHandler\Error\
OutOfMemoryError
in /home/delta/public_html/libraries/vendor/joomla/database/src/Mysqli/MysqliStatement.php (line 581)

Warning: session_write_close(): Cannot call session save handler in a recursive manner in /home/delta/public_html/libraries/vendor/joomla/session/src/Storage/NativeStorage.php on line 114

Warning: session_write_close(): Failed to write session data using user defined save handler. (session.save_path: /var/cpanel/php/sessions/ea-php74) in /home/delta/public_html/libraries/vendor/joomla/session/src/Storage/NativeStorage.php on line 114

---------------------------------
Forum Post Assistant (v1.6.5) : 19-Nov-2022 wrote:
Last PHP Error(s) Reported :: wrote:[19-Nov-2022 01:32:12 UTC] PHP Warning: session_write_close(): Failed to write session data using user defined save handler. (session.save_path: /var/cpanel/php/sessions/ea-php74) in /home/delta/public_html/libraries/vendor/joomla/session/src/Storage/NativeStorage.php on line 114
Basic Environment :: wrote:Joomla! Instance :: Joomla! 4.2.4-Stable (Uaminifu) 25-October-2022
Joomla! Configured :: Yes | Read-Only (444) |
Configuration Options :: Offline: true | SEF: true | SEF Suffix: false | SEF ReWrite: true | .htaccess/web.config: Yes | GZip: true | Cache: true | CacheTime: 15 | CacheHandler: file | CachePlatformPrefix: false | FTP Layer: false | Proxy: false | LiveSite: | Session lifetime: 240 | Session handler: database | Shared sessions: false | SSL: 2 | Error Reporting: maximum | Site Debug: true | Language Debug: false | Default Access: Public | Unicode Slugs: false | dbConnection Type: mysqli | PHP Supports J! 4.2.4: Yes | Database Supports J! 4.2.4: Yes | Database Credentials Present: Yes |

Host Configuration :: OS: Linux | OS Version: 2.6.32-042stab123.2 | Technology: x86_64 | Web Server: Apache | Encoding: gzip, deflate, br | System TMP Writable: Yes | Free Disk Space : 247.15 GiB |

PHP Configuration :: Version: 7.4.33 | PHP API: fpm-fcgi | Session Path Writable: Yes | Display Errors: 0 | Error Reporting: 32759 | Log Errors To: /home/delta/logs/delta_sawstemp_net.php.error.log | Last Known Error: 19th November 2022 01:32:12. | Register Globals: | Magic Quotes: | Safe Mode: | Allow url fopen: 1 | Open Base: | Uploads: 1 | Max. Upload Size: 64M | Max. POST Size: 64M | Max. Input Time: 60 | Max. Execution Time: 30 | Memory Limit: 768M

Database Configuration :: Version: 5.5.5-10.3.37-MariaDB (Client:mysqlnd 7.4.33) | Database Size: 177.15 MiB | #of Tables with config prefix:  107 | #of other Tables:  0 | User Privileges : GRANT ALL
Detailed Environment :: wrote:PHP Extensions :: Core (7.4.33) | date (7.4.33) | libxml (7.4.33) | openssl (7.4.33) | pcre (7.4.33) | zlib (7.4.33) | filter (7.4.33) | hash (7.4.33) | pcntl (7.4.33) | Reflection (7.4.33) | SPL (7.4.33) | session (7.4.33) | standard (7.4.33) | cgi-fcgi (7.4.33) | bcmath (7.4.33) | calendar (7.4.33) | ctype (7.4.33) | curl (7.4.33) | dom (20031129) | ftp (7.4.33) | gd (7.4.33) | iconv (7.4.33) | imap (7.4.33) | json (7.4.33) | mbstring (7.4.33) | mysqlnd (mysqlnd 7.4.33) | PDO (7.4.33) | Phar (7.4.33) | posix (7.4.33) | SimpleXML (7.4.33) | sockets (7.4.33) | sqlite3 (7.4.33) | tokenizer (7.4.33) | xml (7.4.33) | xmlwriter (7.4.33) | xsl (7.4.33) | zip (1.15.6) | mysqli (7.4.33) | pdo_mysql (7.4.33) | pdo_sqlite (7.4.33) | xmlreader (7.4.33) | Zend Engine (3.4.0) |
Potential Missing Extensions ::
Disabled Functions :: exec | passthru | shell_exec | system |

Switch User Environment :: PHP CGI: No | Server SU: No | PHP SU: No | Potential Ownership Issues: No
Folder Permissions :: wrote:Core Folders :: images/ (755) | components/ (755) | modules/ (755) | plugins/ (755) | language/ (755) | templates/ (755) | cache/ (755) | logs/ (755) | tmp/ (755) | administrator/components/ (755) | administrator/modules/ (755) | administrator/language/ (755) | administrator/templates/ (755) | administrator/logs/ (755) |

Elevated Permissions (First 10) ::
Database Information :: wrote:Database statistics :: Uptime: 965671 | Threads: 11 | Questions: 10094134 | Slow queries: 35 | Opens: 419621 | Flush tables: 1 | Open tables: 1999 | Queries per second avg: 10.452 |
Extensions Discovered :: wrote:Components :: Site ::
Core ::
3rd Party::

Components :: Admin ::
Core :: com_messages (4.0.0) 1 | com_cache (4.0.0) 1 | com_weblinks (4.0.1) 1 | com_modules (4.0.0) 1 | com_search (4.0.0-dev) 1 | com_actionlogs (3.9.0) 1 | com_cpanel (4.0.0) 1 | com_workflow (4.0.0) 1 | com_config (4.0.0) 1 | com_fields (4.0.0) 1 | com_tags (4.0.0) 1 | com_checkin (4.0.0) 1 | com_joomlaupdate (4.0.3) 1 | com_templates (4.0.0) 1 | com_associations (4.0.0) 1 | com_login (4.0.0) 1 | com_privacy (3.9.0) 1 | com_scheduler (4.1.0) 1 | com_contenthistory (4.0.0) 1 | com_media (3.0.0) 1 | com_admin (4.0.0) 1 | com_wrapper (4.0.0) 1 | com_newsfeeds (4.0.0) 1 | com_menus (4.0.0) 1 | com_installer (4.0.0) 1 | com_finder (4.0.0) 1 | com_mails (4.0.0) 1 | com_content (4.0.0) 1 | com_categories (4.0.0) 1 | com_users (4.0.0) 1 | com_redirect (4.0.0) 1 | com_plugins (4.0.0) 1 | com_languages (4.0.0) 1 | com_ajax (4.0.0) 1 | com_postinstall (4.0.0) 1 | com_banners (4.0.0) 1 |
3rd Party:: COM_OSMAP (5.0.6) 1 | COM_COOKIESPOLICYNOTIFICATIONBAR (4.2.5) 1 | com_pixtracker (2.0.2) 1 | com_gantry5 (5.5.14) 1 | COM_JCHOPTIMIZE_PRO (7.3.1) 0 | com_facebookfeedpro (3.7.4) 1 | com_admintools (7.1.11) 1 | Maximenu CK (9.1.27) 1 | com_maximenuck (9.1.27) 1 | com_ampz (4.2.2) 1 | com_akeebabackup (9.4.2) 1 |

Modules :: Site ::
Core :: mod_weblinks (4.0.1) 1 | mod_whosonline (3.0.0) 1 | mod_articles_latest (3.0.0) 1 | mod_tags_popular (3.1.0) 1 | mod_finder (3.0.0) 1 | mod_wrapper (3.0.0) 1 | mod_feed (3.0.0) 1 | mod_custom (3.0.0) 1 | mod_banners (3.0.0) 1 | mod_random_image (3.0.0) 1 | mod_login (3.0.0) 1 | mod_articles_categories (3.0.0) 1 | mod_menu (3.0.0) 1 | mod_stats (3.0.0) 1 | mod_languages (3.5.0) 1 | mod_articles_archive (3.0.0) 1 | mod_users_latest (3.0.0) 1 | mod_search (4.0.0-dev) 1 | mod_articles_news (3.0.0) 1 | mod_breadcrumbs (3.0.0) 1 | mod_footer (3.0.0) 1 | mod_tags_similar (3.1.0) 1 | mod_articles_category (3.0.0) 1 | mod_related_items (3.0.0) 1 | mod_syndicate (3.0.0) 1 | mod_articles_popular (3.0.0) 1 |
3rd Party:: MOD_OSDONATE (2.0.0) 1 | JW Player Module Advanced (5.25.0) 1 | Paypal Donate (2.1.2) 1 | Maximenu CK (9.1.27) 1 | MOD_SHACKICONS (4.0.0) 1 | RAXO All-mode PRO (1.12) 1 | RAXO Default Module Layout (4.1) ? | RAXO Bricks Module Layout (4.1) ? | RAXO List Module Layout (4.1) ? | RAXO List Module Layout (3.2) ? | RAXO Newsbreak Module Layout (1.4) ? | RAXO Columns Module Layout (3.3) ? | RAXO Columns Module Layout (4.1) ? | RAXO Prime Module Layout (1.1) ? | RAXO Default Module Layout (3.3) ? | RAXO Bricks Module Layout (3.3) ? | RAXO All-mode PRO (2.02) 1 | RAXO Default Module Layout (4.1) ? | RAXO Bricks Module Layout (4.1) ? | RAXO List Module Layout (4.1) ? | RAXO List Module Layout (3.2) ? | RAXO Newsbreak Module Layout (1.4) ? | RAXO Columns Module Layout (3.3) ? | RAXO Columns Module Layout (4.1) ? | RAXO Prime Module Layout (1.1) ? | RAXO Default Module Layout (3.3) ? | RAXO Bricks Module Layout (3.3) ? | mod_gantry5_particle (5.5.14) 1 |

Modules :: Admin ::
Core :: mod_logged (3.0.0) 1 | mod_user (4.0.0) 1 | mod_toolbar (3.0.0) 1 | mod_frontend (4.0.0) 1 | mod_feed (3.0.0) 1 | mod_messages (4.0.0) 1 | mod_custom (3.0.0) 1 | mod_submenu (3.0.0) 1 | mod_login (3.0.0) 1 | mod_multilangstatus (3.0.0) 1 | mod_latestactions (3.9.0) 1 | mod_menu (3.0.0) 1 | mod_privacy_dashboard (3.9.0) 1 | mod_version (3.0.0) 1 | mod_quickicon (3.0.0) 1 | mod_post_installation_messages (4.0.0) 1 | mod_loginsupport (4.0.0) 1 | mod_privacy_status (4.0.0) 1 | mod_popular (3.0.0) 1 | mod_title (3.0.0) 1 | mod_stats_admin (3.0.0) 1 | mod_sampledata (3.8.0) 1 | mod_latest (3.0.0) 1 |
3rd Party:: MOD_JCHMODESWITCHER (7.3.1) 0 |

Libraries ::
Core ::
3rd Party:: file_fof40 (4.1.0) ? | file_fof30 (3.6.2) ? |

Plugins ::
Core :: plg_api-authentication_basic (4.0.0) 0 | plg_api-authentication_token (4.0.0) 1 | plg_fields_checkboxes (3.7.0) 1 | plg_fields_text (3.7.0) 1 | plg_fields_usergrouplist (3.7.0) 1 | plg_fields_radio (3.7.0) 1 | plg_fields_url (3.7.0) 1 | plg_fields_user (3.7.0) 1 | plg_fields_color (3.7.0) 1 | plg_fields_imagelist (3.7.0) 1 | plg_fields_textarea (3.7.0) 1 | plg_fields_integer (3.7.0) 1 | plg_fields_media (3.7.0) 1 | plg_fields_subform (4.0.0) 1 | plg_fields_calendar (3.7.0) 1 | plg_fields_sql (3.7.0) 1 | plg_fields_editor (3.7.0) 1 | plg_fields_list (3.7.0) 1 | plg_finder_newsfeeds (3.0.0) 1 | plg_finder_content (3.0.0) 1 | plg_finder_weblinks (4.0.1) 1 | plg_finder_tags (3.0.0) 1 | plg_finder_categories (3.0.0) 1 | plg_finder_contacts (3.0.0) 1 | plg_extension_finder (4.0.0) 1 | plg_extension_joomla (3.0.0) 1 | plg_extension_namespacemap (4.0.0) 1 | plg_behaviour_taggable (4.0.0) 1 | plg_behaviour_versionable (4.0.0) 1 | plg_workflow_publishing (4.0.0) 1 | plg_workflow_notification (4.0.0) 1 | plg_workflow_featuring (4.0.0) 1 | plg_quickicon_extensionupdate (3.0.0) 1 | plg_quickicon_privacycheck (3.9.0) 1 | plg_quickicon_overridecheck (4.0.0) 1 | plg_quickicon_downloadkey (4.0.0) 1 | plg_quickicon_joomlaupdate (3.0.0) 1 | plg_quickicon_phpversioncheck (3.7.0) 1 | plg_user_contactcreator (3.0.0) 0 | plg_user_terms (3.9.0) 0 | plg_user_profile (3.0.0) 0 | plg_user_token (3.9.0) 1 | plg_user_joomla (3.0.0) 1 | plg_privacy_user (3.9.0) 1 | plg_privacy_content (3.9.0) 1 | plg_privacy_consents (3.9.0) 1 | plg_privacy_message (3.9.0) 1 | plg_privacy_actionlogs (3.9.0) 1 | plg_content_fields (3.7.0) 1 | plg_content_vote (3.0.0) 1 | plg_content_finder (3.0.0) 1 | plg_content_pagenavigation (3.0.0) 1 | plg_content_confirmconsent (3.9.0) 0 | plg_content_emailcloak (3.0.0) 1 | plg_content_pagebreak (3.0.0) 1 | plg_content_loadmodule (3.0.0) 1 | plg_content_joomla (3.0.0) 1 | plg_search_newsfeeds (4.0.0-dev) 1 | plg_search_content (4.0.0-dev) 1 | plg_search_weblinks (4.0.1) 1 | plg_search_tags (4.0.0-dev) 0 | plg_search_categories (4.0.0-dev) 1 | plg_search_contacts (4.0.0-dev) 0 | plg_task_site_status (4.1) 1 | plg_task_demo_tasks (4.1) 1 | plg_task_check_files (4.1) 1 | plg_task_requests (4.1) 1 | plg_sampledata_multilang (4.0.0) 1 | plg_filesystem_local (4.0.0) 1 | plg_system_fields (3.7.0) 1 | plg_system_webauthn (4.0.0) 1 | plg_system_updatenotification (3.5.0) 1 | plg_system_languagefilter (3.0.0) 0 | plg_system_accessibility (4.0.0) 0 | plg_system_schedulerunner (4.1) 1 | plg_system_privacyconsent (3.9.0) 0 | plg_system_highlight (3.0.0) 1 | plg_system_redirect (3.0.0) 1 | plg_system_shortcut (4.2.0) 1 | plg_system_sessiongc (3.8.6) 1 | plg_system_sef (3.0.0) 1 | plg_system_httpheaders (4.0.0) 0 | plg_system_logrotation (3.9.0) 1 | plg_system_weblinks (4.0.1) 0 | plg_system_log (3.0.0) 1 | plg_system_skipto (4.0.0) 1 | plg_system_logout (3.0.0) 1 | plg_system_stats (3.5.0) 1 | plg_system_debug (3.0.0) 1 | plg_system_remember (3.0.0) 0 | plg_system_jooa11y (4.2.0) 1 | plg_system_languagecode (3.0.0) 0 | plg_system_task_notification (4.1) 1 | plg_system_cache (3.0.0) 0 | plg_system_actionlogs (3.9.0) 0 | plg_media-action_crop (4.0.0) 1 | plg_media-action_resize (4.0.0) 1 | plg_media-action_rotate (4.0.0) 1 | plg_editors-xtd_fields (3.7.0) 1 | plg_editors-xtd_readmore (3.0.0) 1 | plg_editors-xtd_article (3.0.0) 1 | plg_editors-xtd_weblink (4.0.1) 0 | plg_editors-xtd_menu (3.7.0) 1 | plg_editors-xtd_pagebreak (3.0.0) 1 | plg_editors-xtd_module (3.5.0) 1 | plg_editors-xtd_image (3.0.0) 1 | plg_authentication_ldap (3.0.0) 0 | plg_authentication_cookie (3.0.0) 1 | plg_authentication_joomla (3.0.0) 1 | plg_actionlog_joomla (3.9.0) 1 | plg_multifactorauth_webauthn (4.2.0) 0 | plg_multifactorauth_fixed (4.2.0) 0 | plg_multifactorauth_yubikey (3.2.0) 0 | plg_multifactorauth_email (4.2.0) 0 | plg_multifactorauth_totp (3.2.0) 1 | plg_installer_override (4.0.0) 1 | plg_installer_urlinstaller (3.6.0) 1 | plg_installer_folderinstaller (3.6.0) 1 | plg_installer_webinstaller (4.0.0) 1 | plg_installer_packageinstaller (3.6.0) 1 | plg_webservices_newsfeeds (4.0.0) 1 | plg_webservices_menus (4.0.0) 1 | plg_webservices_users (4.0.0) 1 | plg_webservices_languages (4.0.0) 1 | plg_webservices_privacy (4.0.0) 1 | plg_webservices_redirect (4.0.0) 1 | plg_webservices_content (4.0.0) 1 | plg_webservices_config (4.0.0) 1 | plg_webservices_messages (4.0.0) 1 | plg_webservices_plugins (4.0.0) 1 | plg_webservices_media (4.1.0) 1 | plg_webservices_tags (4.0.0) 1 | plg_webservices_installer (4.0.0) 1 | plg_webservices_banners (4.0.0) 1 | plg_webservices_templates (4.0.0) 1 | plg_webservices_modules (4.0.0) 1 | plg_captcha_recaptcha (3.4.0) 1 | plg_captcha_recaptcha_invisible (3.8) 0 |
3rd Party:: plg_gantry5_preset (5.5.14) 0 | PLG_QUICKICON_AKEEBABACKUP (9.4.2) 1 | plg_quickicon_gantry5 (5.5.14) 1 | PLG_USER_JCHOPTIMIZEUSERSTATE (7.3.1) 0 | plg_content_jwadvanced (4.25.0) 1 | plg_content_pixtracker (2.0.2) 1 | PLG_OSMAP_K2 (5.0.6) 0 | PLG_OSMAP_WEBLINKS (5.0.0) 0 | PLG_OSMAP_JDOWNLOADS (5.0.6) 0 | PLG_OSMAP_PHOCAGALLERY (5.0.6) 0 | PLG_OSMAP_MTREE (5.0.0) 0 | PLG_OSMAP_SOBIPRO (5.0.6) 0 | PLG_OSMAP_OSCAMPUS (5.0.6) 0 | PLG_OSMAP_JOOMLA (5.0.6) 1 | PLG_OSMAP_SHACKLOCATIONS (5.0.6) 0 | PLG_OSMAP_VM (5.0.6) 0 | PLG_OSMAP_KUNENA (5.0.6) 0 | PLG_PWEBOPENGRAPH (3.0.1) 1 | plg_system_pixanalytic (2.0.2) 1 | System - ampz (4.2.2) 1 | plg_system_gantry5 (5.5.14) 1 | plg_system_osmylicensesmanager (2.0.15) 1 | PLG_SYSTEM_JCHOPTIMIZE (7.3.1) 0 | PLG_SYSTEM_WEB357FRAMEWORK (1.9.1) 1 | PLG_SYSTEM_COOKIESPOLICYNOTIFICATIO (4.2.5) ? | PLG_SYSTEM_JCHOPTIMIZEPAGECACHE (7.3.1) 0 | plg_system_ospamanot (2.0.1) 1 | System - Facebook Feed Pro (3.7.4) 1 | PLG_SYSTEM_ADMINTOOLS (7.1.11) 1 | PLG_AJAX_WEB357FRAMEWORK (1.9.1) 1 | PLG_AJAX_COOKIESPOLICYNOTIFICATIONB (4.2.5) ? | PLG_EDITORS-XTD_AMPZ (4.2.2) 1 | PLG_ACTIONLOG_ADMINTOOLS (7.1.11) 0 | plg_editors_tinymce (5.10.5) 1 | plg_editors_codemirror (5.65.6) 1 | PLG_INSTALLER_COOKIESPOLICYNOTIFICA (4.2.5) ? |
Templates Discovered :: wrote:Templates :: Site :: js_impacto (3.6.14) 0 | cassiopeia (1.0) 1 | rt_studius (1.1.0) 1 |
Templates :: Admin :: atum (1.0) 1 |

maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by maestroc » Sat Nov 19, 2022 1:41 am

Also see this in the stack trace:

OutOfMemoryError
Symfony\Component\ErrorHandler\Error\OutOfMemoryError:
Error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 131324784 bytes)

at /home/delta/public_html/libraries/vendor/maximebf/debugbar/src/DebugBar/DataFormatter/DataFormatter.php:41

maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by maestroc » Sat Nov 19, 2022 1:46 am

Unless I turn off the debugger the back end just keeps giving me memory errors after running an index attempt.

User avatar
ceford
Joomla! Ace
Joomla! Ace
Posts: 1496
Joined: Mon Feb 24, 2014 10:38 pm
Location: Edinburgh, Scotland
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by ceford » Sat Nov 19, 2022 6:49 am

Try this from the command line, after changing directory to cli:

php -d memory_limit=512M joomla.php finder:index

Or whatever memory_limit value you prefer.

User avatar
Per Yngve Berg
Joomla! Master
Joomla! Master
Posts: 29650
Joined: Mon Oct 27, 2008 9:27 pm
Location: Romerike, Norway

Re: Smart Search index never completes, mem limit at 400MB

Post by Per Yngve Berg » Sat Nov 19, 2022 10:06 am

I will recommend changing the Session Handler from 'database' to 'filesystem' in Global Configuration to offload the database with Session Data. PHP handles it more effectively than the database.

User avatar
sakiss
Joomla! Explorer
Joomla! Explorer
Posts: 326
Joined: Wed Aug 20, 2008 4:09 pm

Re: Smart Search index never completes, mem limit at 400MB

Post by sakiss » Mon Nov 21, 2022 11:18 am

Try to reduce the "Indexer batch size" setting from the Smart Search Options/Settings

maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by maestroc » Wed Nov 30, 2022 1:59 am

I increased memory limit to 512M
Switched session handler to filesystem
Indexer batch size is set to 5

Still getting the same problem. Please see the screencast video below to see things more clearly. You will see that the indexer makes it to 12 out of 490 articles then stops (same place every time). I let it run but after 15-20 minutes it still says 12 out of 490.

https://screenrec.com/share/MdnUHJwYjB

If I cancel it switches to a red error screen with this out of memory error.
Fatal error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 137591280 bytes) in /home/delta/public_html/plugins/system/debug/src/JavascriptRenderer.php on line 114
OutOfMemoryError
HTTP 500 Whoops, looks like something went wrong.
Error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 137591280 bytes)

Symfony\Component\ErrorHandler\Error\
OutOfMemoryError
in /home/delta/public_html/plugins/system/debug/src/JavascriptRenderer.php (line 114)
$js = $this->getJsInitializationCode(); } if ($renderStackedData && $this->debugBar->hasStackedData()) { foreach ($this->debugBar->getStackedData() as $id => $data) { $js .= $this->getAddDatasetCode($id, $data, '(stacked)'); } } $suffix = !$initialize ? '(ajax)' : null; $js .= $this->getAddDatasetCode($this->debugBar->getCurrentRequestId(), $this->debugBar->getData(), $suffix);

User avatar
ceford
Joomla! Ace
Joomla! Ace
Posts: 1496
Joined: Mon Feb 24, 2014 10:38 pm
Location: Edinburgh, Scotland
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by ceford » Wed Nov 30, 2022 4:56 am

I noticed that your Database Collation is latin1_swedish_ci. Mine are:

Database Collation utf8mb4_unicode_ci
Database Connection Collation utf8mb4_general_ci

Make sure all character columns of all tables are utf8mb4_general_ci

Can you try the command line? From /home/username/siteroot/cli:

Code: Select all

php -d memory_limit=512M joomla.php finder:index
Finder Indexer
==========================

 Starting Indexer
 Setting up Smart Search plugins
 Setup 32 items in 0.02 seconds.
  * Processed batch 1 in 0.172 seconds.
  * Skipping pause, as previous batch had a very low processing time (0.172s < 1s)

Total Processing Time: 0.193 seconds.
Peak memory usage: 16,777,216 bytes

SharkyKZ
Joomla! Hero
Joomla! Hero
Posts: 2508
Joined: Fri Jul 05, 2013 10:35 am
Location: Parts Unknown

Re: Smart Search index never completes, mem limit at 400MB

Post by SharkyKZ » Wed Nov 30, 2022 9:05 am

Out of memory error occurs due to changes introduced in the Debug plugin in 4.2.4. You should report this to developers. In the meantime, you can disable debug or enable "Track Request History" option in the plugin.

User avatar
ceford
Joomla! Ace
Joomla! Ace
Posts: 1496
Joined: Mon Feb 24, 2014 10:38 pm
Location: Edinburgh, Scotland
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by ceford » Wed Nov 30, 2022 10:03 am

ceford wrote:
Wed Nov 30, 2022 4:56 am
Make sure all character columns of all tables are utf8mb4_general_ci
Big mistake/mistype - that should be utf8mb4_unicode_ci

User avatar
sakiss
Joomla! Explorer
Joomla! Explorer
Posts: 326
Joined: Wed Aug 20, 2008 4:09 pm

Re: Smart Search index never completes, mem limit at 400MB

Post by sakiss » Wed Nov 30, 2022 12:41 pm

Please disable the "System Debug" and set the "Error Reporting" to None

maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by maestroc » Thu Dec 01, 2022 4:30 am

With debug off and error reporting on if I run the indexer I get a progress bar that makes it about halfway across then stops with

undefined
Call to undefined method Joomla\CMS\Document\JsonDocument::getHeadData()

If I run the command line as requested I get this:

Finder Indexer
==========================

Starting Indexer
Setting up Smart Search plugins
Setup 508 items in 0.013 seconds.
* Processed batch 1 in 0.031 seconds.
* Skipping pause, as previous batch had a very low processing time (0.031s < 1s)
* Processed batch 2 in 0.006 seconds.
* Skipping pause, as previous batch had a very low processing time (0.006s < 1s)
* Processed batch 3 in 0.005 seconds.
* Skipping pause, as previous batch had a very low processing time (0.005s < 1s)
* Processed batch 4 in 0.015 seconds.
* Skipping pause, as previous batch had a very low processing time (0.015s < 1s)
* Processed batch 5 in 0.01 seconds.
* Skipping pause, as previous batch had a very low processing time (0.01s < 1s)
* Processed batch 6 in 0.008 seconds.
* Skipping pause, as previous batch had a very low processing time (0.008s < 1s)
* Processed batch 7 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 8 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 9 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 10 in 0.008 seconds.
* Skipping pause, as previous batch had a very low processing time (0.008s < 1s)
* Processed batch 11 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 12 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 13 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 14 in 0.011 seconds.
* Skipping pause, as previous batch had a very low processing time (0.011s < 1s)
* Processed batch 15 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 16 in 0.011 seconds.
* Skipping pause, as previous batch had a very low processing time (0.011s < 1s)
* Processed batch 17 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 18 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 19 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 20 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)
* Processed batch 21 in 0.009 seconds.
* Skipping pause, as previous batch had a very low processing time (0.009s < 1s)

In jwadvanced.php line 596:

Call to undefined method Joomla\CMS\Document\RawDocument::getHeadData()


finder:index [--minproctime MINPROCTIME] [--pause PAUSE] [--divisor DIVISOR] [--live-site [LIVE-SITE]] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi][--no-ansi] [-n|--no-interaction] [--] <command> [<purge>]

maestroc
Joomla! Explorer
Joomla! Explorer
Posts: 429
Joined: Sun Dec 04, 2005 7:27 pm
Contact:

Re: Smart Search index never completes, mem limit at 400MB

Post by maestroc » Thu Dec 01, 2022 4:38 am

I figured out that the jwadvanced.php is the JW Player Advanced plugin. I disabled it, ran the indexer and it seems to be working. jwadvanced is supposed to be compatible with J4 so not sure what the deal is...

User avatar
sakiss
Joomla! Explorer
Joomla! Explorer
Posts: 326
Joined: Wed Aug 20, 2008 4:09 pm

Re: Smart Search index never completes, mem limit at 400MB

Post by sakiss » Fri Dec 02, 2022 12:11 pm

As i wrote try to disable both the debug and the error reporting.

If ajax returns errors or warnings it will never finish


Post Reply

Return to “Administration Joomla! 4.x”