加速Plone之降龙十八掌——之三
加速Plone之降龙十八掌——之三
http://www.315ok.org/blogfolder/5
http://www.315ok.org/logo.png
加速Plone之降龙十八掌——之三
加速Plone之降龙十八掌——之三
Quick Recap
● Step 1: Cache your static content in the browser
– Associate files and images in your skins with
HTTPCache
● Use cache=HTTPCache in the .metadata file
● CacheFu will do the rest
– Register your CSS and JS with portal_css/portal_js
● Make them mergeable
● Save when css/js change
● CacheFu will take care of caching
Quick Recap
● Keep limitations in mind
– Only helps if people load the URL more than once!
● Great for CSS / JS / images that appear on all pages
– Once it's on the browser, can't change until it expires
● Unless you are using something cool like RR
Proxy cache
● Benefit of browser cache:
– Every request served by cache is one less request
served by ZServer
● Drawback of browser cache:
– Can't invalidate stale content
● Alternative for content that changes more
frequently: use a proxy cache
Strategy 2: Proxy Caching
● Idea: put a fast but dumb proxy cache in front of Zope
● Proxy cache serves up (static) content, keeps load off Zope
● Zope can tell proxy cache when content expires so you don't serve up stale content
Proxy cache
● Because it is server side, cached content is shared
– Browser cache only helps if 1 client requests same resource twice
– Proxy cache helps if 2 (anonymous) people request same thing even if they are different people
– Much less help when content is personalized, though
● Our strategy: cache anonymous content
● Possible to expand if content is personalized based on,say, roles instead of username
● Will talk more about personalized content later
Plone and content caching
● By default, Plone sends no Cache-Control header,which means that pages won't be cached in general
● Anything using main_template has headers set in
global_cache_headers.pt
– In CMFPlone/skins/plone_templates
● contains Cache-Control: no-cache
– CacheFu overrides, uses caching_policy_manager
instead
Plone and content caching
● Want to override default headers for a single page?
– Simplest way: call request.RESPONSE.setHeader in
body of template.
● Overrides previous header, affects only template in question.
● May get stomped by caching_policy_manager
– Harder way: create a caching_policy_manager policy
● (You won't need to do this in general)
Content cache headers
● Goal is to cache anonymous content views
● Not much point caching personalized views
– Not enough hits per cached page to justify
– Fills up the cache
● How do we control content cache headers?
– With a caching policy, of course
– Content views will use 2 different policies
● cache_in_squid if you are anonymous
● cache_in_memory if you are authenticated
Content cache policies
● Leave content in squid; purge as needed
● cache_in_squid
– max-age = 0
● Don't cache in the browser!
– s-max-age = 86400
● Cache in squid for up to 24 hours
● Keep out of squid
● cache_in_memory
– Don't cache in browser or squid
– max-age = 0, s-max-age = 0
How policies are assigned
● How does Zope know what caching policies to
apply?
– Cache setup tool controls everything: The One Ring
● Integrates the 7 different products
– Nice portlet – let's look
● Site setup -> Cache Configuration Tool
– Main tab controls relationship with squid
● Talk about that later
– Next tabs control policy assignments
Cache configuration tool
● When an object looks for headers, it gets sent to
CacheSetup
● CacheSetup walks through its own policies to
figure out what the appropriate caching policy is
● HTTPCache content
– Assigns all content associated with HTTPCache
– Both anonymous and authenticated users get “Cache in
browser” policy
– Hopefully reasonably self-explanatory
Cache configuration tool
● Next tab: Plone content types
● Have an object + template in hand. Does the policy apply?
– Look at content type – is it in the list?
– Look at template
● Is it a default view for the object?
● Is it on the list of templates?
– Look at request – is there anything that should stop
caching?
Cache configuration tool
● Ok, so the configuration policy applies, now what?
● Need to figure out a caching policy
● 2 methods:
– Use policy specified for anonymous or authenticated users
– Get the policy ID from an external script
● For default views of main Plone content objects:
– cache in squid for anonymous users
– cache in memory for authenticated users
Cache configuration tool
● For default views of main Plone container objects:
– cache in memory for anonymous and authenticated users
● Reason:
– Can purge content objects when they change, BUT
– Container views change when any of their contents
change
● So either all content has to purge parent OR
● Just cache in RAM and work out purging another way (will discuss later)
Caching Your Views
● Recommended method:
– Add a new assignment policy
● In portal_cache_settings, add a new content policy
– Select your content types
– Indicate that default views should be cached
– Choose type of caching policy for anonymous and authenticated
– Configure ETags (will discuss later – default Plone
Etags are good starting point)
Purging
● What happens when content changes?
– CMFSquidTool purges the object
● CacheSetup configures squidtool so you don't have to
– Monkey patches index, unindex, reindex, etc
– When an object is created / modified / deleted, cache is
purged
● Cache configuration tool figures out the right
pages to purge
– Typically just the views and templates specified
– If you want extras, you can add a script
Purging
● Plone content types
– uses script to purge extra pages
● Why?
– If you modify the file “myfile”, need to purge:
● default views: myfile, myfile/, myfile/view
● also myfile/download
– If you modify the image “myimg”, need to purge
● default views: myimg, myimg/, myimg/view
● also myimg/image_thumbnail, etc
● Script supplies the extra /download,
image_thumbnail, etc
Proxy Caches
● Squid
– free, open source; runs on Linux, Windows, OSX
– http://www.squid-cache.org
– Super fast (~150 requests/second on mid-range box)
● Some (but probably not all) of CacheFu strategy
should work with IIS + Enfold Enterprise Proxy
– http://www.enfoldsystems.com/Products/EEP
Why not Apache?
● Apache + mod_cache
– Lots of documentation about using Apache for caching
● Problem: mod_cache doesn’t support purging
– No easy way to delete stale pages from cache
● Should be possible to modify CacheFu to get some (but not full)
benefit from Apache
– 1-2 days work
– Sponsorship welcome!
Using Squid
● Excellent documentation available
● (Only need to read a few chapters, not whole book)
Using Squid
● Squid has a reputation of being complex
● Problem is that default squid.conf is 3500 lines
– 99% documentation
– most options don't apply
● CacheFu contains sample squid.conf
– 137 lines (including comments)
– straightforward to configure
● CacheFu has sample configurations for
– squid by itself
– squid behind Apache
● useful if you need to wire together different web apps and
want to use mod_rewrite, etc
– setup is similar
● Pick the appropriate setup
Configuring squid
● Go to the directory for the configuration you have
chosen
– squid_direct or squid_behind_apache
● Edit squid.conf and follow the instructions
– Walkthrough
● Edit redirector_class.py and set up the redirection
rules
– Syntax is like mod_rewrite for Apache
– Walkthrough
Setting up squid
● Copy everything (squid.conf, all .py files) to
/etc/squid
● Fire up squid!
Setting up squid
● Tips:
– Check file permissions
● squid must have read access to squid.conf, iRedirector.py,
squidAcl.py, and redirector_class.py
● squid must have execute access to iRedirectory.py and
squidAcl.py
– squidAcl.py and iRedirectory.py get called directly
● First line is #!/usr/local/bin/python -Ou
● If your python is not at /usr/local/bin/python, change the path to python in the first lines of these files
● Make sure you can run both of these from the command line without getting an exception
● Step 1: Cache your static content in the browser
– Associate files and images in your skins with
HTTPCache
● Use cache=HTTPCache in the .metadata file
● CacheFu will do the rest
– Register your CSS and JS with portal_css/portal_js
● Make them mergeable
● Save when css/js change
● CacheFu will take care of caching
Quick Recap
● Keep limitations in mind
– Only helps if people load the URL more than once!
● Great for CSS / JS / images that appear on all pages
– Once it's on the browser, can't change until it expires
● Unless you are using something cool like RR
Proxy cache
● Benefit of browser cache:
– Every request served by cache is one less request
served by ZServer
● Drawback of browser cache:
– Can't invalidate stale content
● Alternative for content that changes more
frequently: use a proxy cache
Strategy 2: Proxy Caching
● Idea: put a fast but dumb proxy cache in front of Zope
● Proxy cache serves up (static) content, keeps load off Zope
● Zope can tell proxy cache when content expires so you don't serve up stale content
Proxy cache
● Because it is server side, cached content is shared
– Browser cache only helps if 1 client requests same resource twice
– Proxy cache helps if 2 (anonymous) people request same thing even if they are different people
– Much less help when content is personalized, though
● Our strategy: cache anonymous content
● Possible to expand if content is personalized based on,say, roles instead of username
● Will talk more about personalized content later
Plone and content caching
● By default, Plone sends no Cache-Control header,which means that pages won't be cached in general
● Anything using main_template has headers set in
global_cache_headers.pt
– In CMFPlone/skins/plone_templates
● contains Cache-Control: no-cache
– CacheFu overrides, uses caching_policy_manager
instead
Plone and content caching
● Want to override default headers for a single page?
– Simplest way: call request.RESPONSE.setHeader in
body of template.
● Overrides previous header, affects only template in question.
● May get stomped by caching_policy_manager
– Harder way: create a caching_policy_manager policy
● (You won't need to do this in general)
Content cache headers
● Goal is to cache anonymous content views
● Not much point caching personalized views
– Not enough hits per cached page to justify
– Fills up the cache
● How do we control content cache headers?
– With a caching policy, of course
– Content views will use 2 different policies
● cache_in_squid if you are anonymous
● cache_in_memory if you are authenticated
Content cache policies
● Leave content in squid; purge as needed
● cache_in_squid
– max-age = 0
● Don't cache in the browser!
– s-max-age = 86400
● Cache in squid for up to 24 hours
● Keep out of squid
● cache_in_memory
– Don't cache in browser or squid
– max-age = 0, s-max-age = 0
How policies are assigned
● How does Zope know what caching policies to
apply?
– Cache setup tool controls everything: The One Ring
● Integrates the 7 different products
– Nice portlet – let's look
● Site setup -> Cache Configuration Tool
– Main tab controls relationship with squid
● Talk about that later
– Next tabs control policy assignments
Cache configuration tool
● When an object looks for headers, it gets sent to
CacheSetup
● CacheSetup walks through its own policies to
figure out what the appropriate caching policy is
● HTTPCache content
– Assigns all content associated with HTTPCache
– Both anonymous and authenticated users get “Cache in
browser” policy
– Hopefully reasonably self-explanatory
Cache configuration tool
● Next tab: Plone content types
● Have an object + template in hand. Does the policy apply?
– Look at content type – is it in the list?
– Look at template
● Is it a default view for the object?
● Is it on the list of templates?
– Look at request – is there anything that should stop
caching?
Cache configuration tool
● Ok, so the configuration policy applies, now what?
● Need to figure out a caching policy
● 2 methods:
– Use policy specified for anonymous or authenticated users
– Get the policy ID from an external script
● For default views of main Plone content objects:
– cache in squid for anonymous users
– cache in memory for authenticated users
Cache configuration tool
● For default views of main Plone container objects:
– cache in memory for anonymous and authenticated users
● Reason:
– Can purge content objects when they change, BUT
– Container views change when any of their contents
change
● So either all content has to purge parent OR
● Just cache in RAM and work out purging another way (will discuss later)
Caching Your Views
● Recommended method:
– Add a new assignment policy
● In portal_cache_settings, add a new content policy
– Select your content types
– Indicate that default views should be cached
– Choose type of caching policy for anonymous and authenticated
– Configure ETags (will discuss later – default Plone
Etags are good starting point)
Purging
● What happens when content changes?
– CMFSquidTool purges the object
● CacheSetup configures squidtool so you don't have to
– Monkey patches index, unindex, reindex, etc
– When an object is created / modified / deleted, cache is
purged
● Cache configuration tool figures out the right
pages to purge
– Typically just the views and templates specified
– If you want extras, you can add a script
Purging
● Plone content types
– uses script to purge extra pages
● Why?
– If you modify the file “myfile”, need to purge:
● default views: myfile, myfile/, myfile/view
● also myfile/download
– If you modify the image “myimg”, need to purge
● default views: myimg, myimg/, myimg/view
● also myimg/image_thumbnail, etc
● Script supplies the extra /download,
image_thumbnail, etc
Proxy Caches
● Squid
– free, open source; runs on Linux, Windows, OSX
– http://www.squid-cache.org
– Super fast (~150 requests/second on mid-range box)
● Some (but probably not all) of CacheFu strategy
should work with IIS + Enfold Enterprise Proxy
– http://www.enfoldsystems.com/Products/EEP
Why not Apache?
● Apache + mod_cache
– Lots of documentation about using Apache for caching
● Problem: mod_cache doesn’t support purging
– No easy way to delete stale pages from cache
● Should be possible to modify CacheFu to get some (but not full)
benefit from Apache
– 1-2 days work
– Sponsorship welcome!
Using Squid
● Excellent documentation available
● (Only need to read a few chapters, not whole book)
Using Squid
● Squid has a reputation of being complex
● Problem is that default squid.conf is 3500 lines
– 99% documentation
– most options don't apply
● CacheFu contains sample squid.conf
– 137 lines (including comments)
– straightforward to configure
● CacheFu has sample configurations for
– squid by itself
– squid behind Apache
● useful if you need to wire together different web apps and
want to use mod_rewrite, etc
– setup is similar
● Pick the appropriate setup
Configuring squid
● Go to the directory for the configuration you have
chosen
– squid_direct or squid_behind_apache
● Edit squid.conf and follow the instructions
– Walkthrough
● Edit redirector_class.py and set up the redirection
rules
– Syntax is like mod_rewrite for Apache
– Walkthrough
Setting up squid
● Copy everything (squid.conf, all .py files) to
/etc/squid
● Fire up squid!
Setting up squid
● Tips:
– Check file permissions
● squid must have read access to squid.conf, iRedirector.py,
squidAcl.py, and redirector_class.py
● squid must have execute access to iRedirectory.py and
squidAcl.py
– squidAcl.py and iRedirectory.py get called directly
● First line is #!/usr/local/bin/python -Ou
● If your python is not at /usr/local/bin/python, change the path to python in the first lines of these files
● Make sure you can run both of these from the command line without getting an exception