Archive

Archive for the ‘Performance and Tuning’ Category

Fine Tuning Your SharePoint 2010 Environment – Front End Optimizations

June 8, 2010 4 comments

Abstract: Tuning your SharePoint environment is an important step in optimizing its performance and enabling an efficient deployment that reduces overutilization in resources and in turn provides the best possible user experience. In this session, Mike Watson and Shannon Bray will demonstrate techniques to improve the capacity of your SharePoint 2010 environment and discuss ideas and solutions to include software boundaries, caching options, and remote BLOB storage.

TechED 2010: North America gave me the opportunity to co-present a break out session with Mike Watson on SharePoint 2010 performance and capacity planning. Included here are notes to my portion of the presentation along with the steps required to complete the demos. Mike and I broke up the presentation into two distinct areas. I started out with “Frontend Optimizations” and Mike wrapped up with “Backend Optimization”.

This session will be hosted on-line via TechED Live and will be available for FREE!!!

 

 

The word capacity may mean different things to each of us, so before we dig much deeper, let’s take a second to make sure we are all on the same page. According to Dictionary.com, the word ‘Capacity’ means:

 
 

“Actual or potential ability to perform, yield, or withstand”

 
 

 
 

So what does this mean when it comes to SharePoint? In order to find the actual ability of your environment, you need to be able to answer questions similar to the following.

 

How much hardware do we need?

Should we implement a server farm?

Do we need SQL Server?

How much data can we store?

How many users can our environment support?

How many sites can we run on our servers?

How do we validate our design?

 
 

So, why are these questions important to capacity? Knowing how much hardware you need may seem obvious, if you think in terms of data storage. But Capacity is much more than just storage. Let take a look at the components of Capacity.

 

Capacity has three primary components. They include end-user latency, throughput, and storage. I will be focusing on the first two. Let’s start with “End-User Latency”. Microsoft has recently released a new white paper entitled “Capacity Management and Sizing for Microsoft SharePoint Server 2010″. I recommend anyone who is serious about capacity for SharePoint to review it.

 

“The measure of time delay experienced in a system.”

 
 

Latency can be defined as the measure of time delay experienced in a system. This is the time measured from the time the button is clicked to the time the page is rendered. On a great system, this can be less than a second, but we have all experienced those not so great systems. You know the ones that it takes 5, 10, or 15 seconds to bring back results.

Latency is composed of three major components:

  • The time it takes the server to receive and process the request
  • The time it takes the request and the server response to transfer over the network
  • The time it takes the response to render on the client application

 
 

Microsoft has made many investments in dealing with latency. Some of the improvements include: lighter and faster pages, early rendering, IE8 is WAN optimized as compared to IE7, and they have also decreased the page load times of PLT1 and PLT2.

 
 

Office 2010 now has a local file cache which resides on the user’s computer; this is called the Office Document Cache (ODC). When a user opens and document from SharePoint the document is downloaded into this cache and then opened from there. When a user saves the document back to SharePoint it is saved into the document cache and then uploaded in the background (asynchronously) to SharePoint (like email leaving your outbox and going into sent items without you knowing). On top of both this download and upload scenario, only file differentials are sent across the wire, so if only 1 paragraph has changed, it is essentially just that paragraph which is sent on the wire, not the whole document.  There are a few benefits to this ODC model:

 
 

  1. A user can open a document previously cached if the SharePoint server is not available e.g. offline
  2. Reduced network utilization which improves both performance and costs.
  3. When a user saves a document back to SharePoint, it seems like it has happening immediately and they get control back of the application immediately, in the background the document is uploaded to the server. Thus having a GREAT user experience.

 
 

This uses a new protocol Microsoft has put together called FSSHTTP (File Sync via SOAP over HTTP). There are some important points to note about this though:

  1. It only works with Office 2010 on the client and SharePoint 2010 on the server. This is because it requires new components on the client and server to deal with FSSHTTP
  2. It only works with the XML based Office file formats i.e. docx, xlsx, pptx etc

 

http://msdn.microsoft.com/en-us/library/dd956775(v=office.12).aspx

 
 

Mobile Pages in SharePoint – Recommended for high-latency users (above 300 ms RT) and/or narrow bandwidth

 

 

As a SharePoint Architect, you may need to pin point latency issues. Some of the recommended tools include Fiddler, Team System 2010, Virtual Round Trip Analyzer, and the JS Profiler located in IE8.”

 
 

Let’s take a look at what some of these tools have to offer.

The first tool we are going to take a look at is the IE8 JS Profiler. This feature helps developers find performance issues. To see this tool in action, simply load up your SharePoint site and hit F12. You have several options including the opportunity to start a profiler service. One of the tools I like the best is the image report that shows all of the images and their sizes on a page.

The next tool we are going to review is Fiddler. Fiddler is free to download and is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP(S) traffic, set breakpoints, and “fiddle” with incoming or outgoing data.

 

“The quantity or amount of raw material processed within a given time”

 
 

Throughput is the quantity or amount of raw material processed within a given time. In SharePoint 2007, it was not uncommon for a user’s actions to affect the entire server. Deleting a list that has 300,000 items in it, could take a very long time to complete, and while SharePoint is performing this task, other operations suffer.

  
 

I mentioned earlier that Microsoft made several investments in Latency. They have also improved throughput in SharePoint 2010. They have offered this by optimizing SQL Server to sustain higher loads and have added protection to reduce latency spikes inside the software itself.

 
 

Here are some of the key investments they have made.

 

 

 

 

 

 

 

 

We will start off with the Logging Database. If you haven’t seen this yet, I would encourage you to take a look at this as it collects data from across your farm. It is basically a one stop shop or the health and usage data. The database is not on by default. In order to enable it, visit ‘Configure web analytics and health data collection’ and click ‘Enable usage data collection’.

 

 

We can now go into SQL Server and open the logging database.

 

 

 

 

 
 

Next, we have the Developer Dashboard. This is very similar to the Trace.axd file inside ASP.NET that allows a web developer to see key aspects of the page life cycle and the cost of the page. This is now available through the SharePoint UI. The developer dashboard is enabled by the Farm Administrator either through STSADM or PowerShell. It has three states: On, Off, and OnDemand. It can be turned on through Visual Studio, PowerShell, or by using STSADM.

 

 

 

 
 

<Large List Throttling>

 
 

Microsoft has also made some key investments on excessive client load. Here, the server will inform clients on its health condition, thereby allowing the clients to tune down sync frequency. SharePoint 2010 throttles low-priority requests and becomes more aggressive before going down.

 

http://blogs.technet.com/b/speschka/archive/2009/10/27/working-with-large-lists-in-sharepoint-2010-list-throttling.aspx

 

 

 

 

 

Now that we have taken a look at the components of capacity, let’s turn our attention to how latency and throughput factor into common capacity issues. You will notice a solid mix of both hardware and software problems. While software may be the biggest culprit in the form of poorly written code, access control lists on code inside the bin directory, and round trips back and forth to the server; network issues should not be overlooked. We will now take a look at fine tuning your environment.

 

Some of these may seem like no brainers, but you would be surprised at the number of systems that don’t take these into account.

 

We often view minimum requirements as a soft value. This can be due to several factors. I have a laptop with 8 GB of RAM, dual-core, running Server 2008 R2. All of my servers are virtualized using Hyper-V. The system runs pretty good, but I am still under the minimum requirements as you will see from the first line.

Number and speed of round trips to the authentication provider

Authentication provider processing performance

Anonymous

Kerberos

NTLM

Basic

Forms

 
 

Network security can also affect how your system performs. The authentication mechanism used in your environment has an incremental effect on the overall performance of the system. Factors which contribute to authentication performance include the number and speed of the round trips to the authentication provider and the authentication provider processing performance.

 
 

Microsoft’s tests indicate that the order of authentication mechanisms, from fastest to slowest, is: Anonymous, Kerberos, NTLM, Basic, and then Forms. So, what is missing from this list?

 
 

That’s right… Claims based authentication will allow SharePoint to incorporate an authentication model that works with any corporate identity system, including Active Directory, LDAP v2-based directories, application -specific databases, and new user-centric identity models. Each model is different, so it is difficult to actually rank it in this list.

 

Network configuration is critical to the performance of your SharePoint installation. Common network components which can affect your performance include:

  • Network Interface Card (NIC)

    • NIC settings   Where possible, you should always use Gigabit network cards. If you have self-switching cards (100 MB / 1 GB), you should always set the over-ride to use 1 Gigabit.
    • Inbound/Outbound   For scenarios where you expect high traffic, we recommend you have separate NICs to handle inbound and outbound traffic.
  • Switches   If you run your network through a switch, ensure that you are using a GB switch and that you have the same number of inbound/outbound channels.
  • Routers   Ensure your routers are configured on a GB infrastructure.
  • Domain controllers   It is possible for authentication to become a performance bottleneck in your SharePoint environment if the domain controller (DC) receives requests more quickly than it can respond. For environments using user authentication such as NTLM, Microsoft recommends a ratio of 3 Web servers per DC. If your tests indicate that the authentication load at 3 Web servers per DC is acceptable, you can add one more Web server per DC for a supported limit of 4 Web servers per DC.

     
     

    Keep in mind that network configuration should be planned and tested thoroughly prior to moving a system to a production environment.

 
 

So now that we have gotten the network out of the way, let’s move on limits within our favorite Microsoft product.

Here, you will notice that we have two areas to examine. The first is Software Scalability. This is the recommendation for acceptable performance based on software behavior and characteristics. We will discuss some of the common ones here in a moment.

 
 

Secondly, we have hardware scalability. These are the boundaries that do not change software behavior, but can increase overall throughput of a server farm and might be necessary to achieve acceptable performance as the number of objects approach recommended limits.

 
 

So, as many of you may be aware, Microsoft has improved many of the software boundaries from the previous product.

 

Improved

100 Million Items per Search Index (1 Billion with Fast)

Tens of Millions of Documents / Items in a single list

View/Query 5000 items at a time

Remains

150,000 Site Collections per WebApp

50, 000 Site Collections per Content DB

100 GB Content DB (W/R)

 
 

Search has increased the number of items on a Search index from 50 million to 100 million. Not only that, you can now have more than one Search index, so the scalability here is amazing. If you are using FAST, that number increases to a billion.

 
 

The previous limit for a SharePoint list was estimated at 5 Million. In SharePoint 2010, this number has increased to tens of millions.

 
 

One of the most notable improvements is the View/Query items at a time. In SharePoint 2007, it was estimated around 2000. In SharePoint 2010, this number has increase to 5000.

 
 

Some of the other cast of characters have remained the same. We still have a limit of 150,000 Site Collections per web application, 50,000 Site Collections per content database, and a 100 GB limit for a content database that supports proportional reading and writing. You can actually scale up to over a terabyte if your database is mostly read, so the 100 GB size is more of a science than a hard fact. In a recent report by Microsoft, it is actually stated that the size of the database they support is now 200 GB, but the largest a site collection can be is 100 GB. This is available in “SharePoint Server 2010 Capacity Management: Software Boundaries and Limits”

 
 

So let’s take a look at some of our other limits.

A service application provides a resource that can be shared across sites within a farm or, in some cases, across multiple farms. According the preliminary tests, Microsoft does not have a recommended limit for the number of service applications in a single farm.

 
 

In IIS 7.0, an application pool is a group of one or more URLs that are served by a worker process or set of worker processes. When you create a site collection and services in SharePoint, you select an application pool to use or you can create a new application pool. Each app pool has its own worker process and can have a separate security account which prevents two processes from interacting.

 
 

The memory overhead of an application pool is 30 – 50 megabytes plus any memory for the applications running in the application pool process space. The various application demands usually drive the memory usage to over 800 megabytes. While the limit of the number of application pools is influenced by available memory, the general guideline for acceptable performance is to use ten or less.

 
 

You can have up to 5 zones per web application.

 
 

You should have less than 300 content databases per web application

 
 

You should have less than 50,000 site collections per content database

 
 

You should have less than 250,000 sites per site collection.

 
 

Host-named site collections are an option if you want to create multiple root-level site collections within a web application. You can create host-named site collections by using PowerShell. Additionally, by using PowerShell, you can use managed paths with host-named site collections. (New-SPManagedPath -HostHeader)

 
 

Host-named site collections give you more control over URLS. However, host-named site collections are only available through the default zone. User accounts that are configured to authenticate other zones cannot access host-named site collections.

 
 

In SharePoint 2010, host-named sites collections support off-box SSL termination for the protocol scheme. (http:// or https://)

 
 

You can create up to 100,000 host-named site collections within a single IIS Web site.

 

 

You would like to keep your _Total processor time between 50% – 75%.

 
 

Your processor queue length should kept below two times the number of core CPUs

 
 

Memory actually has two key areas: Available Memory and Pages per second. You need to target keeping 20% of the total physical RAM free while keeping the pages per sec below 100.

The most common cause of poor performance in earlier releases of SharePoint Server is the development and deployment of inefficient custom features on top of the SharePoint platform. When developing customer features for SharePoint, there are a number of performance metrics you should monitor. These include, but are not limited to:

 
 

  • SQL Server round trips   For core pages, we recommend no more than 2-3 SQL round trips. Excessive round trips have the following deleterious effect on performance:
    • Increased end user response time due to greater server-side processing time
    • Reduced overall system throughput due to additional load on the database server.
  • SQL server CPU utilization   In order for your MOSS system to remain healthy, it is important that CPU utilization on the database server(s) remains relatively low. If SQL Server 2005 CPU usage averages more than 60%, performance will be adversely affected. Steps you can take to reduce SQL CPU utilization include:
    • Implement a caching strategy – this reduces the overall number of calls from the Web server(s) to the database server.
    • Optimize custom code to use object methods which return your desired data in the most efficient manner (e.g. introduce indexes on lists, etc.)
    • Distribute your SQL databases across multiple physical database servers
  • Page download size   Keep code size to a minimum. A relatively small increase in page size can have a significant impact on performance if that page is accessed by a large number of people every day, especially during peak hours.
  • Client-side code efficiency   Approximately 50% of end user response time is comprised of client side processing of returned code. If your custom solution increases any of these, you can expect an adverse effect on end user response time.
  • AJAX callbacks   For AJAX parts, the number of callbacks, and the payload for each callback. For example, each KPI makes 3 calls in order to return the result. Make sure you test page performance when you introduce multiple KPIs or other custom code into a page.

 
 

You should design your code or performance using best practices and try to re-use existing client code versus adding more. As with any custom code, you should profile your solution so that you understand the potential limitations.

Microsoft SharePoint Server 2010 provides three types of caches that help improve the speed at which web pages load in the browser: The BLOB cache, the page output cache, and the object cache. The BLOB cache is enabled and configured in the Web.config file in the web application to which you want to apply the cache. The changes you make to the Web.Config file will be applied to all of the site collections and sites within the Web application. The page output cache and object cache are usually configured in the user interface at the site collection level; however certain settings for these caches can be configured at the Web application level.

The BLOB cache is stored directly on the hard disk drive of the front-end Web server. The first time that a page is called, these files are copied from the database to the cache on the server hard disk drive, and all subsequent requests for those files are then served from the hard disk drive cache of the server. By default, the BLOB cache is off and must be enabled.

 
 

We will take a look at how to configure this in our upcoming demo.

The page output cache stores the rendered output of a page. It also stores different versions of the cached page, based off of permission of the users who are requesting the page. Page output cache settings can be configured at the site collection level, at the site level, and for page layouts. By default, the page output cache is turned off.

 
 

The page output cache uses cache profiles that specify how long items should be held in the cache. You can specify different cache profiles to be used for anonymous and authenticated users, which optimizes the use of the cache based on the authentication methods that are allowed on the site.

 
 

You can configure cache profiles settings for a Web application by editing the web.config file on the application server. The cache profile settings that you configure at the Web application level will be used for all cache profiles in the site collections for that Web application.

The object cache reduces the amount of traffic between the Web server and the SQL database by storing object – such as lists and libraries , site settings, and page layouts – in memory on the WFE server. As a result, the pages that require these items are able to be rendered quickly, increasing the speed with which pages are delivered to the client browser. Object cache settings can be configured at the Web application level, and at the site collection level. By default, the object cache is on at the site collection level.

 
 

You can optimize the object cache for a web application by specifying the size of the object cache. Specifying a larger number can enhance performance for some large sites at the cost of memory on each front-end Web server. You can configure other settings for the object cache at the site collection level.

 
 

To see an example of how this works, let’s take a look at the SharePoint request cycle.

 

For more detail on SharePoint and the HTTP request, check out the following blog article: http://shannonbray.wordpress.com/2010/04/04/in-the-beginning-sharepoint-2010-and-the-http-request/

 
 

 
 

If your sites use lots of bandwidth, or if you want to use bandwidth more effectively, enable compression to provide faster transmission times between IIS and compression-enabled browsers. If your network bandwidth is restricted, as it is, for example, with mobile phones, compression can improve performance.

IIS provides the following compression options:

  • Static files only
  • Dynamic application responses only
  • Both static files and dynamic application responses

 
 

Compression of dynamic application responses can affect CPU resources because IIS does not cache compressed versions of dynamic output. If compression is enabled for dynamic responses and IIS receives a request for a file that contains dynamic content, the response that IIS sends is compressed every time it is requested. Because dynamic compression consumes significant CPU time and memory resources, use it only on servers that have slow network connections but that have CPU time to spare.

 
 

Unlike dynamic responses, compressed static responses can be cached without degrading CPU resources

 

 

Digging Deeper into the 2010 Caching Options: Part 2

April 4, 2010 2 comments

As I mentioned in my previous post, the object and output cache options require you activate the ‘SharePoint Publishing Feature’ or as I like to call it, the Cha-Ching Feature, as it is normally where Microsoft puts its entire set cool features (i.e. caching, navigation, etc.).

So now that we have experimented a little with the BLOB cache, let’s move on to the output cache. Once you active the appropriate feature(s), you will find your caching options under the Site Administration and Site Collection Administration option of your SharePoint’s setting page.

The output cache is responsible for storing the output of a page. It has the capability to store various versions of the same page based off of parameters. An example of this would be caching pages based off of local area like zip codes. Like the other caching options, output cache can really improve the overall performance of your SharePoint implementation by eliminating the most expensive part of the request: the content database. In a previous post, I outlined the HTTP request. If you examine the steps, you will notice that having output cache enabled cuts out the majority of the hard work help to ensure that the end user is getting what they need without any latency. All though output cache can really help in all areas, it is extremely powerful when it comes to publishing pages with anonymous access enabled. In the world of caching, you don’t want to server your content contributors cached pages, since you want them to be able to review the latest and greatest. When it comes to anonymous users, it’s an entirely different story. Output cache stores the pages in RAM, so after the page is requested the first time, it is ready to rock and roll. The short story is… cache for anonymous and not for publishers. Show how do you separate the two? Caching profiles help you do just that, and in SharePoint 2010, they have improved quite a bit.

Once you click the ‘Output Cache Settings’ link, you will have the option to select the ‘Enable output cache’ checkbox. Under this you will notice the Default Page Output Cache Profile.

A cache profile specifies how long items should be held in the cache. It also describes to the caching system how to determine whether a cached page element is in fact valid for other requests for the same element from different users.

You can specify different cache profiles to use for anonymous and authenticated users. This optimizes the use of the cache based on the authentication methods allowed on the site.

Page output cache profiles specifically affect portal publishing pages.

So by default, you have four caching profiles to choose from: Disabled, Public Internet, Extranet, and Intranet.

You also have the opportunity to specify a different caching policy for page layout designers and administrators and also specify whether or not you wish to collect Debug Cache Information. The debug cache information shows up in the source of your web page and will let you know which caching profile is currently enabled and when the information was saved.

Now we are going to take a look at a caching profile so that you can see how these guys work. We will focus on the Public Internet profile. Here you will see various properties that you can set. TechNet has an article on how to set many of the cache settings, but doesn’t go into caching profiles, so I will discuss these a bit.

Perform ACL Check – When this is set to ‘yes’, it will ensure that all of the items in the cache are properly trimmed for security by checking the access control list of the items. That sounds like something you might want to do all of the time, but remember this is an anonymous profile, and these visitor have limited to no rights.

Duration – This is how long the item will be cached for in seconds. The default setting is 180, so it’s 3 minutes.

Check for Changes – If this is set to ‘Yes’, SharePoint will check for changes in the page prior to rendering. So if you have this set, the duration doesn’t really add much value. If you want to keep your content for a specific amount of time, set this to ‘No’ and use the duration to its fullest potential.

Now we get to the Vary by parameters. These allow you to store multiple versions of the same page based off of specific values. You may have one cache setting for one HTTP Header versus another or based off of a Query String Parameter. Query Strings are represented in the URL by way of values after the ‘?’.

The Cacheability parameter specifies the devices that the object can be cached on. These are an enumerations are from HTTPCacheability (ASP.NET 2.0) and has six values, four that map directly to a Cache-Control HTTP header settings and two special values called HttpCacheability.ServerAndPrivate and HttpCacheability.ServerAndNoCache. ServerAndPrivate, the default setting here, allows a response to be cached only on the origin server and the requesting client. ServerAndNoCache means only the origin server.

Safe for Authenticated Use specifies whether or not an administrator can use this policy for authenticated users.

Allow writers to view cached content does exactly what it says. I don’t know of a good reason to use this one, but someone must have one, J

Ok, so that’s the run down on the caching profiles. Mix that with the output caching and you have a pretty powerful tool at your disposal.

Most of the caching options are only available to the Site Collection Administrator, but the Site Administrator does get the Site Output Cache option, which simply allows for overriding the Site Collection cache profile settings.

In my research for really trying to understand all of the caching mechanisms, I found a post called Measuring ASP.NET and SharePoint output cache. It’s short and sweet, so I will be applying these concepts to my virtual farm and make it the topic of my next post.

Digging Deeper into the 2010 Caching Options: Part 1

April 4, 2010 5 comments

There is nothing complicated about caching in SharePoint 2010. TechNet provides a decent overview of the three options available that will improve the overall performance of how the web pages load in the browser. If you haven’t read the article on Cache Settings Operations, I would definitely encourage you to do so. SharePoint stores most of the content it needs to serve up in the content database, so any time we can save a trip, we are helping the performance of our application.

So we will be examining the three types: BLOB cache, page output, and object cache. Page output and object cache require that you activate the ‘SharePoint Server Publishing’ feature located in the site features of your web site, so these will not be available to you if you are using SharePoint Foundation. BLOB caching, on the other hand, is available in the server product and will be where we start off.

 

BLOB Caching is a disk based cache. You configure it in the web.config file of the web site, which means that you will need to go to each WFE to set it up. Your web.config file is typically located at Local Disk:\inetpub\wwwroot\wss\VirtualDirectories\SiteFolder. Before making any changes to your web.config, you will need to back it up. It can be quite difficult to repair this file if you happen to make a mistake.

Inside the web.config file, find the following line:

<BlobCache location=”C:\BlobCache\14″ path=”\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$” maxSize=”10″ enabled=”false” />

If you are familiar with BlobCache from 2007, it works the same way. One nice thing about 2010, is that you get a very long list of file types that default to Blob cache as opposed to the previous version. I am going to save much of the explanation of Blob caching for the article written on TechNet: Plan for caching and performance. Once you have it set up, upload a few photos to a picture library and will see them populate the blob location.

In the root of your blob location, you will find a series of folders. Each of these folders represents the various web applications on your WFE. If you’re a little confused on which folder maps to which web site, simply check the Sites in your IIS Manager. The folder name is the Site ID.

 

In each of the sub folders is an additional folder and a file named currentcache.bin. If you rename the currentcache.bin file to a text file and open it up, you will see that it is the path referencing the cryptic named folder associated with it. Inside the cryptic folder, you will start to see the site structure of some of your content and additional bin files: change.bin, dump.bin, fulshcount.bin, filestodelete.bin, inprogressdump.bin, and oldcachefolders.bin.

change.bin: This file contains serialized change tokens (SPChangeToken) for objects being cached in the local file system.  These tokens allow the BlobCache maintenance thread to query the content source(s) and subsequently update the contents of the BLOB cache with any items that are identified as having changed since the last maintenance sweep.

dump.bin: This file contains a serialized copy of the BlobCache’s cache dictionary.  The dictionary maintains information for all objects being tracked and maintained by the BlobCache object; each key/value pair in the dictionary consists of a local file path (key) and its associated BlobCacheEntry (value).

flushcount.bin: This file contains nothing more than the serialized value of the cacheFlushCount for the BlobCache object.  Practically speaking, this value allows a BlobCache to determine if a flush has been requested while it was shutdown.

 

One of the changes from MOSS 2007 and SP2010, is that the files use to be stored with a .cache extension. They now have their original description. Another change that I noticed is that the ‘Disk Based Cached Reset’ is no longer available, but you are able to flush the object cache.

 

One thing I was expecting is that if I went into the file system and deleted the files from the cache directly, that they would be populated on the next request. This was not the case. Once deleted, the files were still visible on the site. I was thinking that SharePoint was now going back to the content database to get these file, but I was wrong. I started up an instance of Fiddler to see if I could find these requests. These files are now being rendered by the cache of my web browser. When I deleted my temporary files, I found that the image no longer rendered. I tried several different ways to repair my site. I used the ‘Object Cache Flush’ to see if it had any influence on the BLOB cache. It did not fix my issue. In fact, I had to go as far as deleting all of the .bin files and toggle the web.config file to get the image to render again with blob cache enabled.

 

In MOSS 2007, we had a property called max-age. It allowed you to specify how long cached content would be valid in you blob location. This was measured in seconds, so I attempted to enter a value of “60″ to force my image to go back to the database. I wanted to make sure there weren’t any leftover references, so I once again deleted the .bin files, my image and toggled my web.config. The image was recreated in the blob location with a time stamp of 2:11 PM. I expected to wait a few minutes and refresh the page and get a new item in the blob storage location. This was not the case.

I did find some interesting behavior while playing around with the image properties. I found that if you have an image stored in the blob location and then change one of the properties, the image was updated leading me to believe that the max-age no longer works, but the SharePoint team has obviously set flags to check if the file has any changes and then refreshes the file accordingly. You may also notice that your image may not render until your browser cache is cleared.

In my previous post, I mentioned the HTTPModules that get included in the HTTP request. While researching BLOB cache a little deeper, I found a very informative post by Sean McDonough, that states that BLOB cache is implemented by way of Microsoft.SharePoint.Publishing. This class is loaded in the PublishingHTTPModule. This means that while Blob cache does not require that publishing features be activated, it will not work in SharePoint Foundation. I went back to review the list of IIS modules to see where the PublishHTTPModule fell into the ordered list view. I have it highlighted in blue below.

 

In the next post, we will take a look at the output cache.

In the Beginning: SharePoint 2010 and the HTTP Request

April 4, 2010 4 comments

 
 

In the beginning, there was an HTTP request. Prior to getting back that first glance of your new SharePoint 2010 Team site, you have to initiate a request to a web front end (WFE) that is hosting the site, but what actually happens during this request? Coming from the ASP.NET world into SharePoint, I always have had an elementary understanding of how the typically HTTP request worked. I recently decided to really study many of the aspects of how SharePoint 2010 work under the covers, and why not start at the beginning.

For those of you who have an understanding of how the HTTP request worked in SharePoint 2007 with IIS 6.0, you may be surprised to learn that there have been a few changes, but not all of these changes are actually being used my SharePoint 2010.

When you submit a HTTP request, the request is intercepted by the HTTP listener (HTTP.sys), this is true in both IIS 6.0 and 7.0. The HTTP listener is responsible for both receiving the response and sending it back to the requesting client. IIS 7.0 enables kernel-mode by authentication by default and runs under the machine account. Using kernel-mode has some benefits are far as caching and request queuing are concerned and can improve authentication performance. Kernel-mode also prevents authentication problems with application pools configured to use a custom identity.

In my SharePoint environment, I examined the SharePoint Central Administration Authentication settings. There you will notice that ASP.NET Impersonation and Windows Authentication are enabled. If you click on ‘Advanced Settings’ in the right pane, you will notice that by default ‘Enable Kernel-mode authentication’ is not selected by default.


According to Chun Liu, SharePoint 2010 does not use Kernel-mode for two primary reasons:

  1. There is an incompatibility between IIS7 kernel mode authentication and IE8 which is the level 1 browser for SharePoint 2010. IE8 uses a way so-called “pre-authentication” to avoid some authentication round trips. But when working with IIS7 kernel mode authentication, the IE8 user may experience separate authentication prompt for every single resource on the first request after IIS recycled. (BugID: Windows 7 628823)
  2. Kernel mode authentication does not support the new Nego2 authentication in IIS7.5.

One of the biggest changes from IIS 6.0 to IIS 7.0 is a change in how the HTTP listener works. In IIS 6.0, the WWW Service managed HTTP administration and configuration, process management, and performance monitoring. The worker process in IIS 7.0 has now moved to a new service called the Windows Process Activation Service (WAS). So this means that the WWW Services takes care of the administration and configuration part of the request and then notifies WAS when a request enters the request queue. The WAS allows you to use the same configuration and process model for HTTP and non-HTTP sites. According to a recent post by Pete Harris, you can run WAS without WWW Service if you don’t need HTTP functionality. Both the WAS and WWW Service run as LocalSystem in the same Svchost.exe process and share the same binaries:

Another difference between IIS 6 and IIS 7, is that IIS now keeps the majority of the functionality in modules, which are components that you can added and removed. This allows you to trim down the HTTP request by only using what SharePoint needs.

So let’s take a deeper look at the modules used by SharePoint 2010. If you open the web.config of your Central Administration site and search for the modules section inside <system.webServer> tag. IIS 7.0 has two types of modules. We have the native module and the managed module. Native modules are the ones that come with IIS, while the managed modules are used to extend IIS functionality. Some of the common native modules include: CustomErrorModule, HTTPRedirectionModule, FileAuthorization, AnonymousIdentification, Profile, etc.

Here are a list of the caching modules that are native to IIS 7.0:

Module Name

Description

Resource

FileCacheModule

Provides user mode caching for files and file handles.

Inetsrv\Cachfile.dll

HTTPCacheModule

Provides kernel mode and user mode caching in HTTP.sys.

Inetsrv\Cachhttp.dll

TokenCacheModule

Provides user mode caching of user name and token pairs for modules that produce Windows user principals.

Inetsrv\Cachtokn.dll

UriCacheModule

Provides user mode caching of URL information.

Inetsrv\Cachuri.dll

 
 

I mention these here, as later we will list out the steps in the HTTP request. Since these are native IIS, they will be present in the HTTP pipeline unless they are removed. If we examine the web.config file, we will notice the following modules either being removed or added.

 
 

Removed

Added

AnonymousIdentification

SPRequestModule

FileAuthorization

ScriptModule

Profile

SharePoint14Module

WebDAVModule

StateServiceModule

Session

Session

  

PublishingHTTPModule

 
 

For a list of native modules, you can use the IIS 7.0 Manager.  Simply select the web application in the left pane, and then double click ‘Modules’ in the center pane.


 
 

“In a worker process, an HTTP request passes through several ordered steps, called events, in the Web Server Core. At each event, a native module processes part of the request, such as authenticating the user or adding information to the event log. If a request requires a managed module, the native ManagedEngine module creates an AppDomain, where the managed module can perform the necessary processing, such as authenticating a user with Forms authentication. When the request passes through all of the events in the Web Server Core, the response is returned to HTTP.sys. Figure 2, below, shows an HTTP request entering the worker process.”

 
 

So armed with this information, we should be able to put the SharePoint HTTP request together:

HTTP Request

  • User issues a request over the internet
  • The HTTP Protocol Stack (HTTP.sys) intercept the request and passes it to the Svchost.exe
  • Svchost.exe contains the Windows Activation Services (WAS) and the WorldWide Web Publishing Service (WWW Service)
  • WAS requests configuration information from the configuration store, applicationHost.config. This will map virtual directories to physical directories.
  • WWW Service receives configuration information, such as application pool and site configuration
  • WWW Service uses the configuration information to configure HTTP.sys
  • WAS starts a worker process for the application pool to which the request was made
  • Authentication is handled.
  • Request comes in for an ASP.NET Page.
  • The request is given to the SharePoint14Module.
  • The page handler is unable to find a precompiled version of the page class on disk, so it must grab the file and give it to the ASP.NET engine for parsing.
  • The SharePoint14Module will contact either the file system or the content database for the appropriate page class information.
  • The ASP.NET Page engine parses the file and generates a page class.
  • The page class is compiled into a .NET assembly and cached on the disk.
  • An instance of the requested page’s class is created.
  • The response generated from the requested page class is sent back to the original caller.

 
 

Subsequent Requests not Using Output Caching

  • Request comes in for an ASP.NET Page.
  • The page handler finds a cached version of the class and creates an instance.
  • The response generated from the requested page class is sent back to the caller.

 
 

Requests Using Output Caching

  • Request comes in for an ASP.NET Page.
  • Steps 1-16 are repeated for the first request.
  • The generated response is added to the ASP.NET Cache.
  • The response is sent back to the caller.

 
 

Subsequent Requests Using Output Caching

  • Request comes in for an ASP.NET Page.
  • When the request is processed by the Http Modules, the Cache module is passed through. If the page response is found within the ASP.NET Cache, the response is copied from the Cache API and sent straight back to the caller.

 
 

 
 

 
 

 
 

 
 

 
 

  

Follow

Get every new post delivered to your Inbox.

Join 30 other followers