Varnish cache - Joomla login issues - session

Discussion regarding Joomla! 3.x Performance issues.

Moderator: General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Windows Defender SmartScreen Issues <-- please read this if using Windows 10.
tescojim
Joomla! Apprentice
Joomla! Apprentice
Posts: 18
Joined: Thu Oct 11, 2007 10:03 am
Contact:

Varnish cache - Joomla login issues - session

Post by tescojim » Thu Nov 27, 2014 5:43 pm

Hi,

I found a fantastic guide to setup Varnish cache with Joomla. The problem is that it is not a fully functional Varnish cache configuration.

So here is the code:

Code: Select all

# Place the following 2 configuration blocks right after your "backend default {…}" block
# inside your /etc/varnish/default.vcl file (the main Varnish configuration file)

# This Varnish configuration makes use of a custom HTTP header to determin whether
# some user is logged in or not inside Joomla! To allow this, simply append this code
#    	// Set user state in headers
#		if (!$user->guest) {
#			JResponse::setHeader('X-Logged-In', 'True', true);
#		} else {
#			JResponse::setHeader('X-Logged-In', 'False', true);
#		}
# on the "function onAfterInitialise(){ ... } function, right after "$user= JFactory::getUser();"
# in the Joomla! cache plugin php file located at /plugins/system/cache/cache.php
# Finally, enable this plugin via the Joomla! backend.
# If you don't want to use the cache plugin, add this code in your template's index.php file.
# Don't forget to prepend it with "$user = JFactory::getUser();"

# The following setup assumes a 5 min cache time - you can safely drop this to 1 min for popular/busy sites

sub vcl_recv {

    # Forward client's IP to backend
	remove req.http.X-Forwarded-For;
	set req.http.X-Forwarded-For = client.ip;

	# Proxy (pass) any request that goes to the backend admin,
	# the banner component links or any post requests
    # You can add more pages or entire URL structure in the end of the "if"
	if(req.http.cookie ~ "userID" || req.url ~ "^/administrator" || req.url ~ "^/component/banners" || req.request == "POST") {
		return (pass);
	}
	
	# Check for the custom "x-logged-in" header to identify if the visitor is a guest,
	# then unset any cookie (including session cookies) provided it's not a POST request
	if(req.http.x-logged-in == "False" && req.request != "POST"){
		unset req.http.cookie;
	}

	# Properly handle different encoding types
	if (req.http.Accept-Encoding) {
	  if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
	    # No point in compressing these
	    remove req.http.Accept-Encoding;
	  } elsif (req.http.Accept-Encoding ~ "gzip") {
	    set req.http.Accept-Encoding = "gzip";
	  } elsif (req.http.Accept-Encoding ~ "deflate") {
	    set req.http.Accept-Encoding = "deflate";
	  } else {
	    # unknown algorithm (aka crappy browser)
	    remove req.http.Accept-Encoding;
	  }
	}

	# Cache files with these extensions
	if (req.url ~ "\.(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
		return (lookup);
	}

	# Set how long Varnish will cache content depending on whether your backend is healthy or not
	if (req.backend.healthy) {
		set req.grace = 5m;
	} else {
		set req.grace = 1h;
	}

	return (lookup);
}

sub vcl_fetch {
	
	# Check for the custom "x-logged-in" header to identify if the visitor is a guest,
	# then unset any cookie (including session cookies) provided it's not a POST request
	if(req.request != "POST" && beresp.http.x-logged-in == "False") {
		unset beresp.http.Set-Cookie;
	}
	
	# Allow items to be stale if needed (this value should be the same as with "set req.grace"
	# inside the sub vcl_recv {…} block (the 2nd part of the if/else statement)
	set beresp.grace = 1h;
	
	# Serve pages from the cache should we get a sudden error and re-check in one minute
	if (beresp.status == 503 || beresp.status == 502 || beresp.status == 501 || beresp.status == 500) {
	  set beresp.grace = 60s;
	  return (restart);
	}
	
	# Unset the "etag" header (suggested)
	unset beresp.http.etag;
	
	# This is Joomla! specific: fix stupid "no-cache" header sent by Joomla! even
	# when caching is on - make sure to replace 300 with the number of seconds that
	# you want the browser to cache content
	if(beresp.http.Cache-Control == "no-cache" || beresp.http.Cache-Control == ""){
		set beresp.http.Cache-Control = "max-age=300, public, must-revalidate";
	}
	
	# This is how long Varnish will cache content
	set beresp.ttl = 5m;
	
	return (deliver);
}
Source: https://snipt.net/fevangelou/the-perfec ... -websites/

I can confirm that this Varnish config works quite well apart from login.
As soon as we apply this config users can't login on frontend. No login and no logout.
It seems to me that is messes up the session process.

I also use memcache which is ok, but Varnish overrides the Joomla session handling.
My question is here that is there any chance to "fix" or modify this Varnish config and don't let Varnish intervention in the session process?

It would be a really big help.

Thanks
Signature rules: Literal URLs only - http://forum.joomla.org/viewtopic.php?f=8&t=65

User avatar
jisse
Joomla! Intern
Joomla! Intern
Posts: 80
Joined: Sat Aug 20, 2005 2:13 pm
Location: Netherlands
Contact:

Re: Varnish cache - Joomla login issues - session

Post by jisse » Mon Dec 01, 2014 9:13 am

Hi,

Looking at the configuration, I can see that there is a cookie name 'userID' defined that tells whether a Joomla user is logged in or not. However, this cookie is not part of the Joomla core, so you should be using a Joomla plugin to add this cookie whenever a login needs to occur. Did you make sure to install such a plugin?

I found this on the JED: http://extensions.joomla.org/extensions ... ache/28387
Did you check this extension out already?
Jisse Reitsma
founder of Yireo / author of Programming Joomla Plugins book
www.yireo.com

tescojim
Joomla! Apprentice
Joomla! Apprentice
Posts: 18
Joined: Thu Oct 11, 2007 10:03 am
Contact:

Re: Varnish cache - Joomla login issues - session

Post by tescojim » Mon Dec 01, 2014 10:39 am

Hi jisse,

No. This is only a Varnish cache configuration file. It does not reqire any additional Joomla plugin however I may have missed this bit in the decs of this config file:

Code: Select all

# This Varnish configuration makes use of a custom HTTP header to determin whether
# some user is logged in or not inside Joomla! To allow this, simply append this code
#       // Set user state in headers
#      if (!$user->guest) {
#         JResponse::setHeader('X-Logged-In', 'True', true);
#      } else {
#         JResponse::setHeader('X-Logged-In', 'False', true);
#      }
If you think this would be the problem could you tell me where to place and what exactly to make this work?

Thanks
Signature rules: Literal URLs only - http://forum.joomla.org/viewtopic.php?f=8&t=65

User avatar
jisse
Joomla! Intern
Joomla! Intern
Posts: 80
Joined: Sat Aug 20, 2005 2:13 pm
Location: Netherlands
Contact:

Re: Varnish cache - Joomla login issues - session

Post by jisse » Fri Dec 05, 2014 6:25 pm

Hi,

It sounds like you are missing part of the logic. When a visitor accesses the Joomla site, a cookie is created (plus an equivalent server session). The only thing Varnish will be able to monitor is this cookie: But unfortunately, the cookies that Joomla sets tell nothing to external systems (like Varnish) about the state of that session - whether that session was for a guest or an user. To allow Varnish to know about that state (logged in or not logged in), something else needs to be added to Joomla: Most commonly a simple plugin that adds a new cookie or sets an additional HTTP header. I personally would prefer a plugin that sets an extra header.
Jisse Reitsma
founder of Yireo / author of Programming Joomla Plugins book
www.yireo.com

tescojim
Joomla! Apprentice
Joomla! Apprentice
Posts: 18
Joined: Thu Oct 11, 2007 10:03 am
Contact:

Re: Varnish cache - Joomla login issues - session

Post by tescojim » Fri Dec 05, 2014 6:36 pm

Meantime I made Varnish work with Joomla with installing mod_expires and mod_headers Apache modules and add this code to my htaccess:

Code: Select all

Header set Cache-Control "max-age=29030400, public"
...now the problem is that I can't login on frontend nor backed.
Furthermore I tried "Expires Header" plugin but it doesnt work with Falang.
Once I turned on "Expires Header" plugin all my Falang menu translation has gone missing.
Signature rules: Literal URLs only - http://forum.joomla.org/viewtopic.php?f=8&t=65

tescojim
Joomla! Apprentice
Joomla! Apprentice
Posts: 18
Joined: Thu Oct 11, 2007 10:03 am
Contact:

Re: Varnish cache - Joomla login issues - session

Post by tescojim » Sat Dec 06, 2014 4:15 pm

so I was doing some testing with the following varnish config file:

Code: Select all

# Place the following 2 configuration blocks right after your "backend default {…}" block
# inside your /etc/varnish/default.vcl file (the main Varnish configuration file)

# This Varnish configuration makes use of a custom HTTP header to determin whether
# some user is logged in or not inside Joomla! To allow this, simply append this code
#    	// Set user state in headers
#		if (!$user->guest) {
#			JResponse::setHeader('X-Logged-In', 'True', true);
#		} else {
#			JResponse::setHeader('X-Logged-In', 'False', true);
#		}
# on the "function onAfterInitialise(){ ... } function, right after "$user= JFactory::getUser();"
# in the Joomla! cache plugin php file located at /plugins/system/cache/cache.php
# Finally, enable this plugin via the Joomla! backend.
# If you don't want to use the cache plugin, add this code in your template's index.php file.
# Don't forget to prepend it with "$user = JFactory::getUser();"

# The following setup assumes a 5 min cache time - you can safely drop this to 1 min for popular/busy sites

sub vcl_recv {

    # Forward client's IP to backend
	remove req.http.X-Forwarded-For;
	set req.http.X-Forwarded-For = client.ip;

	# Proxy (pass) any request that goes to the backend admin,
	# the banner component links or any post requests
    # You can add more pages or entire URL structure in the end of the "if"
	if(req.http.cookie ~ "userID" || req.url ~ "^/administrator" || req.url ~ "^/component/banners" || req.request == "POST") {
		return (pass);
	}
	
	# Check for the custom "x-logged-in" header to identify if the visitor is a guest,
	# then unset any cookie (including session cookies) provided it's not a POST request
	if(req.http.x-logged-in == "False" && req.request != "POST"){
		unset req.http.cookie;
	}

	# Properly handle different encoding types
	if (req.http.Accept-Encoding) {
	  if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
	    # No point in compressing these
	    remove req.http.Accept-Encoding;
	  } elsif (req.http.Accept-Encoding ~ "gzip") {
	    set req.http.Accept-Encoding = "gzip";
	  } elsif (req.http.Accept-Encoding ~ "deflate") {
	    set req.http.Accept-Encoding = "deflate";
	  } else {
	    # unknown algorithm (aka crappy browser)
	    remove req.http.Accept-Encoding;
	  }
	}

	# Cache files with these extensions
	if (req.url ~ "\.(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf)$") {
		return (lookup);
	}

	# Set how long Varnish will cache content depending on whether your backend is healthy or not
	if (req.backend.healthy) {
		set req.grace = 5m;
	} else {
		set req.grace = 1h;
	}

	return (lookup);
}

sub vcl_fetch {
	
	# Check for the custom "x-logged-in" header to identify if the visitor is a guest,
	# then unset any cookie (including session cookies) provided it's not a POST request
	if(req.request != "POST" && beresp.http.x-logged-in == "False") {
		unset beresp.http.Set-Cookie;
	}
	
	# Allow items to be stale if needed (this value should be the same as with "set req.grace"
	# inside the sub vcl_recv {…} block (the 2nd part of the if/else statement)
	set beresp.grace = 1h;
	
	# Serve pages from the cache should we get a sudden error and re-check in one minute
	if (beresp.status == 503 || beresp.status == 502 || beresp.status == 501 || beresp.status == 500) {
	  set beresp.grace = 60s;
	  return (restart);
	}
	
	# Unset the "etag" header (suggested)
	unset beresp.http.etag;
	
	# This is Joomla! specific: fix stupid "no-cache" header sent by Joomla! even
	# when caching is on - make sure to replace 300 with the number of seconds that
	# you want the browser to cache content
	if(beresp.http.Cache-Control == "no-cache" || beresp.http.Cache-Control == ""){
		set beresp.http.Cache-Control = "max-age=300, public, must-revalidate";
	}
	
	# This is how long Varnish will cache content
	set beresp.ttl = 5m;
	
	return (deliver);
}
It would work flawlessly if (!)...

I added to my template the suggested code:

Code: Select all

// Set user state in headers for Varnish
$user = JFactory::getUser();
if (!$user->guest) {
JResponse::setHeader('X-Logged-In', 'True', true);
} else {
JResponse::setHeader('X-Logged-In', 'False', true);
}
But it didnt work at all so I changed it to this:

Code: Select all

// Set user state in headers for Varnish
$user = JFactory::getUser();
if($user->id > 0){
JResponse::setHeader('X-Logged-In', 'True', true);
} else {
JResponse::setHeader('X-Logged-In', 'False', true);
}
With the modified code in my template I have half success.
When someone is NOT logged in I see the

Code: Select all

X-Logged-In:False
in my HTTP header:
Image

...but once I logged in there is nothing. No FALSE or no TRUE in the HTTP header.
Image

I think this could be the problem why I still can't login if I used Varnish.
If I turn off Varnish then login works like a charm.
Please... anyone... help me to make this work.
Is anything wrong with Varnish configuration file or with the code in my template?
I really would like to get this working. Im sure many Joomla users would find this useful.
Signature rules: Literal URLs only - http://forum.joomla.org/viewtopic.php?f=8&t=65

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Mon May 18, 2015 1:02 pm

as i check this interesting issue i think that you must unset the cookies for login logout and com_users
http://www.stardothosting.com/blog/2011 ... th-joomla/
Also i think that there is another extension can do that
http://extensions.joomla.org/browse/new ... for-joomla
This extention exclude components and menu items if you have somewhere conflict
also this extension has option to clear varnish cache
chat room spontes : http://www.spontes.com

mattnthat
Joomla! Fledgling
Joomla! Fledgling
Posts: 1
Joined: Thu Aug 06, 2015 4:57 am

Re: Varnish cache - Joomla login issues - session

Post by mattnthat » Thu Aug 06, 2015 5:02 am

I think you are very very close to having a working configuration.

Obviously your not hitting the "$user->id > 0" section of your if statement.

Another option could be to look for the precense of the logged in cookie. If cookies (and Set-Cookies) are being removed for everything but POST requests then when a user logs in (which will be a POST) they will have their cookie added via that POST request.
Once the cookie is there then dont respond from cache.

We have used this patter here: https://www.section.io in order to provide Varnish to sites that dont support it out of the box

fotisevangelou
Joomla! Ace
Joomla! Ace
Posts: 1423
Joined: Sun Jan 22, 2006 6:27 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by fotisevangelou » Tue Aug 11, 2015 2:48 pm

1. Use my revised configuration here: https://snipt.net/fevangelou/the-perfec ... -websites/
2. Install K2 so you get the userID cookie set automatically as well as the X-Logged-In headers (no need to modify your template). K2 uses 2 plugins to achieve the desired behaviour unlike some copycat Joomla plugins for Varnish who still haven't got things right. ;)
3. In K2's parameters, add your naked domain under the "Advanced" section in the Cookie Domain option. It's required.
4. Finally, you only need to modify 3 things of my Varnish config file:
a) Adjust the cache time - default is 10 minutes (read the comments for each if statement to figure out which)
b) Add or remove domains to exclude. If you don't care about such a feature, simply delete the 3 related blocks in the last 3 VCL functions.
c) Create specific entry points for user logins. You can create menu items to the user login page, the password reset, the username reminder and so on. Get the SEF URLs for these menu items and place them in the exclusions (inside 2 if statements - one in vcl_recv() and one in vcl_fetch()). Don't use a module in your site with a popup login page that is available across your site. That's a bad practice. Prefer to make a link to some /login page.

The JoomlaWorks website (which is wicked fast) is based on this configuration for Varnish as well as dozens of Joomla sites for which we've improved their performance as part of our upcoming managed hosting service.
Fotis Evangelou / https://www.joomlaworks.net

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Wed Aug 26, 2015 11:05 am

hi all,

Fotis,can you post the correct code,for varnish 4 here?That code is 3 years old,from what people are saying,i don`t think it does work on latest version of varnish witch is : Varnish Cache 4.1.0-tp1.

"I can confirm that this Varnish config works quite well apart from login.
As soon as we apply this config users can't login on frontend. No login and no logout.
It seems to me that is messes up the session process."

With this problem solved out?

The guys from masivescale,sayed that your supporting varnish cache in an way that is wrong,they done some kind of patch : " Joomla doesn't make using Varnish easy, though. By default not only does it send headers that forbid caching, but also starts a session for every anonymous user. While the former is fixable using multiple freely-available plugins, the latter is an issue with the Joomla core. "

Their patch i did not buyed,but it seems pretty good,at least on the demo page,Total time taken to serve 10000 views 1.2 s.

On their demo,the demo works way faster,then joomlaworks website,and that way shoud load any website,i don`t know exactly what they did,and how this patch would effect other things,if you know more about this,maybe someone can get into details.

There must be an solution on joomla core,or with some kind of plugin to solve this kind of troubles.


The thing,is that i don`t feel comfortable to have an patched joomla core,when i update to last version of joomla,the patch will owerwrite again and again right?And mostly their patch,needs again to be applyed and again....
Does your config,solves that problem?And it will work with JFBCONNECT?Without patching joomla?

Except varnish,what about memcache and apc?Did anybody use them ,togheter with varnish ?

If we use varnish,there is no more need of using JOTCACHE component? I can confirm that with that component,and no varnish,i bringed an website from 20 seconds to about 2-3second loading,without CDN.

From what i saw,varnish should be 100 times more faster in joomla,that means 10seconds loading website,should load in 0.1 seconds.

p.s:Jisse,did you actually test this : http://www.smartpixels.net/products/sma ... sh-joomla/ ,with fotis configuration on varnish 4.1 and it does work ?I mean it does solve the login problem?

fotisevangelou
Joomla! Ace
Joomla! Ace
Posts: 1423
Joined: Sun Jan 22, 2006 6:27 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by fotisevangelou » Wed Aug 26, 2015 1:39 pm

The changes to work with Varnish 4.x are minimal.

1. In the vcl_fetch() function change every occurrence of "hit_for_pass" with "pass".
2. In line 44 completely delete the two + signs. String concatenation no longer requires that syntax in version 4.x.

That's it :)
Fotis Evangelou / https://www.joomlaworks.net

fotisevangelou
Joomla! Ace
Joomla! Ace
Posts: 1423
Joined: Sun Jan 22, 2006 6:27 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by fotisevangelou » Wed Aug 26, 2015 1:47 pm

Use APC. Memcache is for uber large deployments (we're talking about millions of visitors per month, complex frontend apps etc.).

As for Varnish, MassiveScale is patching Joomla files and that's ridiculous. Use K2 to create the additional headers and cookie. You don't have to actually use K2. It's just the quickest way until we build a separate extension to make Joomla friendlier to Varnish.

We've done some of the biggest Joomla/Varnish deployments worldwide. We know what we're doing and you do not have to modify any Joomla core files. And you get all this for free ;)

Check these sites for reference and use a browser's dev tools to identify headers and cookies:
http://www.impactwrestling.com/
http://www.gameover.gr/
http://www.thespread.com/
http://www.dikaiologitika.gr/

Don't mind any Cloudflare details you may see, CloudFlare (when used) is used primarily as the DNS provider, it doesn't do dynamic content caching (at least for these sites).
Fotis Evangelou / https://www.joomlaworks.net

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Wed Aug 26, 2015 1:48 pm

So on this code from here : https://snipt.net/fevangelou/the-perfec ... -websites/ ,only those 2 things needs to be remplaced and is working?

And about the login /log out problem ?It will work?Or it will be need it another plugin?

Did you also used jotcache component with Varnish ?Or only varnish ?
Did you used memcache and apc also?

I did not tested yet,but with jotcache without cdn,i did reduced load time from about 20 seconds,to 2-3seconds into an website.
I read that Varnish does an medium of 100x times reduction in joomla.

Maybe you can confirm that result.

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Wed Aug 26, 2015 1:59 pm

I asked this Fotis,because you already worked with this,and you know better.

I personal don`t like to be the core modified,if core is modified,when you update joomla,again and again you have to patch and patch and patch again right ?I mean this is masivescale approach.

So for let`s say until 1milion or 10 milions visitator per month,varnish is enough?

And after you bypass this number,you need to use APC and memcache?

Facebook is using : MEMCACHE +Varnish.

Varnish is more good then LiteSpeed?

I guess you know this component: http://www.jotcomponents.net/web-programming/jotcache .if i install varnish on dedicated server,is no need of that component right?

With it,i obtained from 20 seconds,to 2-3 seconds,but i need way more fast,i need to load in 0.1-0.3 seconds maximum,i guess Varnish can do that,if the website actually loads in 4-5 seconds right?

We definetely need this : " Joomla friendlier to Varnish component. "

Except that,in Octomber,in k2 we will have the pro packages ?and estimated prices?

K2 it will be good,if somehow,archive of news on large website news,if they can keep the archive in cloud like dropbox,or amazon cloud drive or owncloud,anyhow to not have the database overfuelled.
If you can tell me something about this.

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Wed Aug 26, 2015 2:08 pm

p.s:fotis,those websites loads even more better,then joomlaworks ,hahahaha!At least on my side.
I think you did an more good job for them,then you did for your own stuff.

The only thing,is that varnish,will work for the logged in users or not?With the actual setup,and without patching joomla.


Varnish,on dedicated server,should i install on the root ?Or it`s basically installed per cpanel ?

I mean you install it as root or i need to install it as root into the cpanel,that i want to use varnish?

Basically i need to use varnish,just in one website,witch does have an cpanel,and not on all the cpanels.
They load WOW,results are very good :

http://tools.pingdom.com/fpt/#!/dH37Ls/ ... ts-betting
https://gtmetrix.com/reports/www.thespread.com/Rn1OiUoJ

fotisevangelou
Joomla! Ace
Joomla! Ace
Posts: 1423
Joined: Sun Jan 22, 2006 6:27 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by fotisevangelou » Wed Aug 26, 2015 2:51 pm

JoomlaWorks is actually a pretty big site in terms of traffic and it's got lots of logged in visitors in the frontend. And of course the config I've done is built for frontend user interaction, otherwise it wouldn't have the exclusions.

Now, Varnish is possible on cPanel but it would be a pain in the ass to setup.

If you only have one site on that server, prefer to install Ubuntu Server 14.04 64-bit and use something like Tuxlite - http://tuxlite.com/ - to install all the required server component, including Varnish.

You're digging into more advanced territories (mainly the sysadmin's territory) but it's worth it.
Fotis Evangelou / https://www.joomlaworks.net

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Wed Aug 26, 2015 5:44 pm

So apc with varnish will do the job,until i get 1milions visitators per month?

I have centos on my dedicated,but hopefully i think next year i will go with CloudLinux,witch is much more good in terms of security and things that you can do.

Ubuntu from what i know,does not work with WHM,CPANEL and not very easy to deploy stuff.

If the website,will load like those website,it definetely worth it.

Hope you will lunch soon those pro packages and k2 version 3,and after that,maybe you will help to setup the things on the right way on an news portal,i have over 200.000+ news,but the mysql is an pain in the ass right now.

I can only hope that that in k2 ver 3,you did integer some kind of modules,for example k2 works with plugins,and when you ?templatename ,you don`t see for example module position,so you can place for example:3ads units from adsense.
Easyblog5 solve this thing very easily,i think k2 can do the same.

The plugin that exist right now in the market for k2,are very buggy,and are thinked for websites with 10-100 maybe 1000 news,but not with over 100k news.

Hopefully,maybe you do an Varnish component or something,to get things more simple.

I find something :http://thornelabs.net/2015/03/29/setup- ... ancer.html

http://www.liquidweb.com/kb/how-to-inst ... -centos-7/

But basically,after i install how do i run it for the specific website?

It`s an dedicated server,but it does have more cpanel,each cpanel have their own website,so practically i don`t have just one website,there are more websites but each one with their cpanel,to understand more better my configuration.

I know the guys from site ground,implemented and called : SUPER CACHE,or something like that,and can be activated if you buy their hosting,but i`m not interested about their hosting.

I found something for cpanel and whm: https://www.unixy.net/secure/cart.php?a=confproduct&i=1 ,but i don`t know how good is this,

Varnish Script Plugin - cPanel Varnish+Nginx Script - Paid
cPanel Varnish+Nginx Script - One license is for one server

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Thu Aug 27, 2015 9:32 am

fotisevangelou wrote: 2. Install K2 so you get the userID cookie set automatically as well as the X-Logged-In headers (no need to modify your template). K2 uses 2 plugins to achieve the desired behaviour unlike some copycat Joomla plugins for Varnish who still haven't got things right. ;)
Hello foti. i have a question please. if a client don t install K2 how he get the userID cookie?
chat room spontes : http://www.spontes.com

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sat Aug 29, 2015 8:32 am

About login issue in modules after testing we can fix it with bypass cookie. This can be done with this plugin http://extensions.joomla.org/extensions ... for-joomla easy
chat room spontes : http://www.spontes.com

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sat Aug 29, 2015 12:53 pm

i used in my template

Code: Select all

// Set user state in headers for Varnish
$user = JFactory::getUser();
if($user->id > 0){
JResponse::setHeader('X-Logged-In', 'True', true);
} else {
JResponse::setHeader('X-Logged-In', 'False', true);
}
and X-Logged-In works fine, false when i am not login and true when i login
chat room spontes : http://www.spontes.com

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sat Aug 29, 2015 5:19 pm

finally i was wrong with smart varnish plugin. it s not solve the problem with joomla login module when you login
chat room spontes : http://www.spontes.com

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sat Aug 29, 2015 9:32 pm

fotisevangelou wrote:Don't use a module in your site with a popup login page that is available across your site. That's a bad practice. Prefer to make a link to some /login page.
Is there any way to use for example the core module login of joomla? Not pop up, because until now i can t login with no pop up login from frontend
chat room spontes : http://www.spontes.com

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Tue Sep 01, 2015 10:10 am

fotisevangelou wrote: c) Create specific entry points for user logins. You can create menu items to the user login page, the password reset, the username reminder and so on. Get the SEF URLs for these menu items and place them in the exclusions (inside 2 if statements - one in vcl_recv() and one in vcl_fetch()). Don't use a module in your site with a popup login page that is available across your site. That's a bad practice. Prefer to make a link to some /login page.
Hello, is there any possibility to can exclude components or modules?
chat room spontes : http://www.spontes.com

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Sun Sep 13, 2015 2:22 pm

ribo,did you tryed also jotcache ?

so that plugin http://extensions.joomla.org/extensions ... for-joomla ,does not solve the problem ?!

varnish it might be good,but seems that practically needs and LINUX admin,who knows what is doing,in order to setup it right.
varnish is also only for http from my researches,to make it work for https,is possible,but requires more extensive setup with nginx .

I think ribo,as you tested Varnish,you should test jotcache component,and see if the results on time loading is not the same as Varnish.

I wonder if the results are the same,anyhow jotcache,does not work for dynamic content,but it`s an lot more easier to setup,and works with memcached,apc,etc.

Fotis,what do you mean by Uber large app?

I have news website,with k2,around 200k items on k2 items,the table is MYISAM,and it does crash,when i try to repair,i have to stop mysql ,and then repair or from ssh console,else it`s not working.It`s about 1.6GB .

But i have for example piwik,but with inodb,and is 4GB,so much more,and the repair is working.

I hope you lunch soon,those k2 pro packages,as i need help to setup things right on the k2 level.

Is k2 ver 3,able to handle 1M,10M news ?! and for that amount of news,it will be need it memcached from your point of view ?!

As apc is dead,and on php5 we have opcache,i think the right setup will be :

Php5.5.x + OPCODE CACHE+PHPFM +memcached on my side and cloudflare cdn setup right,varnish won`t help me,and is basically not for HTTPS,only http!

Waiting forward to hear from you guys!

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sun Sep 13, 2015 3:07 pm

hello, i tested varnish with apache and not nginx. i also tested without varnish and jotcache.
i can say that varnish with apache is much, much faster.
note, i don t use K2.
what it works: exclude urls that you point as menu items, and you can exclude domains.
what does not work: http://extensions.joomla.org/extensions ... for-joomla not work for login, joomla contact, etc. that you don t exclude url.
another issue with varnish is

Code: Select all

$_SERVER[“REMOTE_ADDR”] 
gives server IP rather than visitor IP
It would be good to exclude components like this http://extensions.joomla.org/extensions ... for-joomla but exclude not work in this extension. also it could be good if we could exclude components in default.vcl
About https i did n t test it. did you test for example to have also req.https instead req.http ?
chat room spontes : http://www.spontes.com

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Sun Sep 13, 2015 3:32 pm

hey ribo,

"i can say that varnish with apache is much, much faster." - okay,i see that is good.

"what does not work: http://extensions.joomla.org/extensions ... for-joomla not work for login, joomla contact, etc. that you don t exclude url." - maybe fotis,can let us know about this,if it`s possible to do it or not with Varnish.

"another issue with varnish is
Code:
$_SERVER[“REMOTE_ADDR”]
gives server IP rather than visitor IP"

varnish is acting like an reverse proxy,mostly like cloudflare is working too.

In order to get the real ip`s ,at least with the cloudflare,normaly in php should be installed mod_cloudflare, maybe for varnish it does exist some mode like that,in order to solve that problem.

To have the server ip,instead of visitator real ip,is an totally mess in terms of security.

Varnish works only with http,so if you have SSL ,into your website or project,it`s an problem.

You need nginx in front of apache and varnish,and to work something like in this schematics: http://edoceo.com/howto/nginx-varnish-ssl :

"Visual Overview

The HTTP requests from the internet hit Nginx, which passes some directly to Apache and some to Varnish (based on rules). Varnish, on cache-miss, will request to Apache. And Apache will likely hit the database. "

Varnish,haves alot of problems and issuess to be easily used with joomla.

As opcache replaces apc,we can try this :
http://stackoverflow.com/questions/1722 ... hp-opcache

I think php 5.5.x with opcache can solve the problems with loading,and works even on https.

Opcache haves also nicely GUI,in order to mange stuff.
In php 5.5 is easy to enable it,and with PHPFM,practically we should get the maximum out of it.

This can be combined with MEMCACHED,but for memcached,i did not find an good GUI,in order to manage the cache.

This is what my server admin tolded me : " Zend Opcache will provide no benefit with your current setup. You'll need PHP-FPM installed on the server in order for PHP processes to use shared memory. That is the benefit to using Zend Opcache, where shared memory between processes allows multiple processes to use the same compiled code.

Right now your server uses suPHP. suPHP forks processes and those processes are very shortlived and exit after less than a couple of seconds. Zend Opcache is beneficial when processes are persistent, which PHP-FPM provides support for.
By keeping compiled versions of your PHP scripts in memory, they do not have to be compiled and are executed immediately."

What is PHP-FPM?
PHP-FPM (FastCGI Process Manager) is an alternative PHP FastCGI implementation with some additional features useful for sites of any size, especially busier sites.

These features include:

Adaptive process spawning (NEW!)
Basic statistics (ala Apache's mod_status) (NEW!)
Advanced process management with graceful stop/start
Ability to start workers with different uid/gid/chroot/environment and different php.ini (replaces safe_mode)
Stdout & stderr logging
Emergency restart in case of accidental opcode cache destruction
Accelerated upload support
Support for a "slowlog"
Enhancements to FastCGI, such as fastcgi_finish_request() - a special function to finish request & flush all data while continuing to do something time-consuming (video converting, stats processing, etc.)


... and much more.

It was not designed with virtual hosting in mind (large amounts of pools) however it can be adapted for any usage model.

If you can do an test with php-fpm +php5.5.x with zend opcache ,and tell us the results,will be very nicely.

I wonder if this will give you more good results,and won`t cause so much troubles.

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sun Sep 13, 2015 4:14 pm

i also did not use apc. php 5.5 with opcache and memcache for session instead database
chat room spontes : http://www.spontes.com

jootify
Joomla! Intern
Joomla! Intern
Posts: 66
Joined: Thu Feb 13, 2014 3:42 pm

Re: Varnish cache - Joomla login issues - session

Post by jootify » Sun Sep 13, 2015 5:34 pm

so you used opcache,php 5.5 and memcached? You used with php-fpm ?or without it?

you used memcached as cache handler and session handler?

I installed bymyself memcached on my server,and asked my web admins to upgrade to php 5.5.x last version,so i can enable opcache.

But i don`t see it in my whm or cpanel.

What gui are you using for opcache ?And what gui are you using for memcached?

Tell me something about memcached working let`s say you have an page with an:News and 30 comments on it dynamic page ajax like facebook pages let`s say,will memcached when an user login ,will be able to display that news and 30 comments ?And if an user post the 31 comment,only basically to cache the new comment to not waste resources ?

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sun Sep 13, 2015 6:05 pm

i use with fcgi , no fpm. memcache use it only for session handler, not for cache handler as opcache works. now i will test mod_remoteip for ip issue
chat room spontes : http://www.spontes.com

User avatar
ribo
Joomla! Virtuoso
Joomla! Virtuoso
Posts: 3507
Joined: Sun Jan 03, 2010 8:47 pm
Contact:

Re: Varnish cache - Joomla login issues - session

Post by ribo » Sun Sep 13, 2015 6:43 pm

i can confirm that mod_remoteip fixes ip issues for apache 2.4 and in

Code: Select all

$_SERVER[“REMOTE_ADDR”]
About memcache and opcache install them from whm and after configure them for your needs
chat room spontes : http://www.spontes.com


Locked

Return to “Performance - Joomla! 3.x”