Home > Performance and Tuning > Fine Tuning Your SharePoint 2010 Environment – Front End Optimizations

Fine Tuning Your SharePoint 2010 Environment – Front End Optimizations


Abstract: Tuning your SharePoint environment is an important step in optimizing its performance and enabling an efficient deployment that reduces overutilization in resources and in turn provides the best possible user experience. In this session, Mike Watson and Shannon Bray will demonstrate techniques to improve the capacity of your SharePoint 2010 environment and discuss ideas and solutions to include software boundaries, caching options, and remote BLOB storage.

TechED 2010: North America gave me the opportunity to co-present a break out session with Mike Watson on SharePoint 2010 performance and capacity planning. Included here are notes to my portion of the presentation along with the steps required to complete the demos. Mike and I broke up the presentation into two distinct areas. I started out with “Frontend Optimizations” and Mike wrapped up with “Backend Optimization”.

This session will be hosted on-line via TechED Live and will be available for FREE!!!

 

 

The word capacity may mean different things to each of us, so before we dig much deeper, let’s take a second to make sure we are all on the same page. According to Dictionary.com, the word ‘Capacity’ means:

 
 

“Actual or potential ability to perform, yield, or withstand”

 
 

 
 

So what does this mean when it comes to SharePoint? In order to find the actual ability of your environment, you need to be able to answer questions similar to the following.

 

How much hardware do we need?

Should we implement a server farm?

Do we need SQL Server?

How much data can we store?

How many users can our environment support?

How many sites can we run on our servers?

How do we validate our design?

 
 

So, why are these questions important to capacity? Knowing how much hardware you need may seem obvious, if you think in terms of data storage. But Capacity is much more than just storage. Let take a look at the components of Capacity.

 

Capacity has three primary components. They include end-user latency, throughput, and storage. I will be focusing on the first two. Let’s start with “End-User Latency”. Microsoft has recently released a new white paper entitled “Capacity Management and Sizing for Microsoft SharePoint Server 2010″. I recommend anyone who is serious about capacity for SharePoint to review it.

 

“The measure of time delay experienced in a system.”

 
 

Latency can be defined as the measure of time delay experienced in a system. This is the time measured from the time the button is clicked to the time the page is rendered. On a great system, this can be less than a second, but we have all experienced those not so great systems. You know the ones that it takes 5, 10, or 15 seconds to bring back results.

Latency is composed of three major components:

  • The time it takes the server to receive and process the request
  • The time it takes the request and the server response to transfer over the network
  • The time it takes the response to render on the client application

 
 

Microsoft has made many investments in dealing with latency. Some of the improvements include: lighter and faster pages, early rendering, IE8 is WAN optimized as compared to IE7, and they have also decreased the page load times of PLT1 and PLT2.

 
 

Office 2010 now has a local file cache which resides on the user’s computer; this is called the Office Document Cache (ODC). When a user opens and document from SharePoint the document is downloaded into this cache and then opened from there. When a user saves the document back to SharePoint it is saved into the document cache and then uploaded in the background (asynchronously) to SharePoint (like email leaving your outbox and going into sent items without you knowing). On top of both this download and upload scenario, only file differentials are sent across the wire, so if only 1 paragraph has changed, it is essentially just that paragraph which is sent on the wire, not the whole document.  There are a few benefits to this ODC model:

 
 

  1. A user can open a document previously cached if the SharePoint server is not available e.g. offline
  2. Reduced network utilization which improves both performance and costs.
  3. When a user saves a document back to SharePoint, it seems like it has happening immediately and they get control back of the application immediately, in the background the document is uploaded to the server. Thus having a GREAT user experience.

 
 

This uses a new protocol Microsoft has put together called FSSHTTP (File Sync via SOAP over HTTP). There are some important points to note about this though:

  1. It only works with Office 2010 on the client and SharePoint 2010 on the server. This is because it requires new components on the client and server to deal with FSSHTTP
  2. It only works with the XML based Office file formats i.e. docx, xlsx, pptx etc

 

http://msdn.microsoft.com/en-us/library/dd956775(v=office.12).aspx

 
 

Mobile Pages in SharePoint – Recommended for high-latency users (above 300 ms RT) and/or narrow bandwidth

 

 

As a SharePoint Architect, you may need to pin point latency issues. Some of the recommended tools include Fiddler, Team System 2010, Virtual Round Trip Analyzer, and the JS Profiler located in IE8.”

 
 

Let’s take a look at what some of these tools have to offer.

The first tool we are going to take a look at is the IE8 JS Profiler. This feature helps developers find performance issues. To see this tool in action, simply load up your SharePoint site and hit F12. You have several options including the opportunity to start a profiler service. One of the tools I like the best is the image report that shows all of the images and their sizes on a page.

The next tool we are going to review is Fiddler. Fiddler is free to download and is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP(S) traffic, set breakpoints, and “fiddle” with incoming or outgoing data.

 

“The quantity or amount of raw material processed within a given time”

 
 

Throughput is the quantity or amount of raw material processed within a given time. In SharePoint 2007, it was not uncommon for a user’s actions to affect the entire server. Deleting a list that has 300,000 items in it, could take a very long time to complete, and while SharePoint is performing this task, other operations suffer.

  
 

I mentioned earlier that Microsoft made several investments in Latency. They have also improved throughput in SharePoint 2010. They have offered this by optimizing SQL Server to sustain higher loads and have added protection to reduce latency spikes inside the software itself.

 
 

Here are some of the key investments they have made.

 

 

 

 

 

 

 

 

We will start off with the Logging Database. If you haven’t seen this yet, I would encourage you to take a look at this as it collects data from across your farm. It is basically a one stop shop or the health and usage data. The database is not on by default. In order to enable it, visit ‘Configure web analytics and health data collection’ and click ‘Enable usage data collection’.

 

 

We can now go into SQL Server and open the logging database.

 

 

 

 

 
 

Next, we have the Developer Dashboard. This is very similar to the Trace.axd file inside ASP.NET that allows a web developer to see key aspects of the page life cycle and the cost of the page. This is now available through the SharePoint UI. The developer dashboard is enabled by the Farm Administrator either through STSADM or PowerShell. It has three states: On, Off, and OnDemand. It can be turned on through Visual Studio, PowerShell, or by using STSADM.

 

 

 

 
 

<Large List Throttling>

 
 

Microsoft has also made some key investments on excessive client load. Here, the server will inform clients on its health condition, thereby allowing the clients to tune down sync frequency. SharePoint 2010 throttles low-priority requests and becomes more aggressive before going down.

 

http://blogs.technet.com/b/speschka/archive/2009/10/27/working-with-large-lists-in-sharepoint-2010-list-throttling.aspx

 

 

 

 

 

Now that we have taken a look at the components of capacity, let’s turn our attention to how latency and throughput factor into common capacity issues. You will notice a solid mix of both hardware and software problems. While software may be the biggest culprit in the form of poorly written code, access control lists on code inside the bin directory, and round trips back and forth to the server; network issues should not be overlooked. We will now take a look at fine tuning your environment.

 

Some of these may seem like no brainers, but you would be surprised at the number of systems that don’t take these into account.

 

We often view minimum requirements as a soft value. This can be due to several factors. I have a laptop with 8 GB of RAM, dual-core, running Server 2008 R2. All of my servers are virtualized using Hyper-V. The system runs pretty good, but I am still under the minimum requirements as you will see from the first line.

Number and speed of round trips to the authentication provider

Authentication provider processing performance

Anonymous

Kerberos

NTLM

Basic

Forms

 
 

Network security can also affect how your system performs. The authentication mechanism used in your environment has an incremental effect on the overall performance of the system. Factors which contribute to authentication performance include the number and speed of the round trips to the authentication provider and the authentication provider processing performance.

 
 

Microsoft’s tests indicate that the order of authentication mechanisms, from fastest to slowest, is: Anonymous, Kerberos, NTLM, Basic, and then Forms. So, what is missing from this list?

 
 

That’s right… Claims based authentication will allow SharePoint to incorporate an authentication model that works with any corporate identity system, including Active Directory, LDAP v2-based directories, application -specific databases, and new user-centric identity models. Each model is different, so it is difficult to actually rank it in this list.

 

Network configuration is critical to the performance of your SharePoint installation. Common network components which can affect your performance include:

  • Network Interface Card (NIC)

    • NIC settings   Where possible, you should always use Gigabit network cards. If you have self-switching cards (100 MB / 1 GB), you should always set the over-ride to use 1 Gigabit.
    • Inbound/Outbound   For scenarios where you expect high traffic, we recommend you have separate NICs to handle inbound and outbound traffic.
  • Switches   If you run your network through a switch, ensure that you are using a GB switch and that you have the same number of inbound/outbound channels.
  • Routers   Ensure your routers are configured on a GB infrastructure.
  • Domain controllers   It is possible for authentication to become a performance bottleneck in your SharePoint environment if the domain controller (DC) receives requests more quickly than it can respond. For environments using user authentication such as NTLM, Microsoft recommends a ratio of 3 Web servers per DC. If your tests indicate that the authentication load at 3 Web servers per DC is acceptable, you can add one more Web server per DC for a supported limit of 4 Web servers per DC.

     
     

    Keep in mind that network configuration should be planned and tested thoroughly prior to moving a system to a production environment.

 
 

So now that we have gotten the network out of the way, let’s move on limits within our favorite Microsoft product.

Here, you will notice that we have two areas to examine. The first is Software Scalability. This is the recommendation for acceptable performance based on software behavior and characteristics. We will discuss some of the common ones here in a moment.

 
 

Secondly, we have hardware scalability. These are the boundaries that do not change software behavior, but can increase overall throughput of a server farm and might be necessary to achieve acceptable performance as the number of objects approach recommended limits.

 
 

So, as many of you may be aware, Microsoft has improved many of the software boundaries from the previous product.

 

Improved

100 Million Items per Search Index (1 Billion with Fast)

Tens of Millions of Documents / Items in a single list

View/Query 5000 items at a time

Remains

150,000 Site Collections per WebApp

50, 000 Site Collections per Content DB

100 GB Content DB (W/R)

 
 

Search has increased the number of items on a Search index from 50 million to 100 million. Not only that, you can now have more than one Search index, so the scalability here is amazing. If you are using FAST, that number increases to a billion.

 
 

The previous limit for a SharePoint list was estimated at 5 Million. In SharePoint 2010, this number has increased to tens of millions.

 
 

One of the most notable improvements is the View/Query items at a time. In SharePoint 2007, it was estimated around 2000. In SharePoint 2010, this number has increase to 5000.

 
 

Some of the other cast of characters have remained the same. We still have a limit of 150,000 Site Collections per web application, 50,000 Site Collections per content database, and a 100 GB limit for a content database that supports proportional reading and writing. You can actually scale up to over a terabyte if your database is mostly read, so the 100 GB size is more of a science than a hard fact. In a recent report by Microsoft, it is actually stated that the size of the database they support is now 200 GB, but the largest a site collection can be is 100 GB. This is available in “SharePoint Server 2010 Capacity Management: Software Boundaries and Limits”

 
 

So let’s take a look at some of our other limits.

A service application provides a resource that can be shared across sites within a farm or, in some cases, across multiple farms. According the preliminary tests, Microsoft does not have a recommended limit for the number of service applications in a single farm.

 
 

In IIS 7.0, an application pool is a group of one or more URLs that are served by a worker process or set of worker processes. When you create a site collection and services in SharePoint, you select an application pool to use or you can create a new application pool. Each app pool has its own worker process and can have a separate security account which prevents two processes from interacting.

 
 

The memory overhead of an application pool is 30 – 50 megabytes plus any memory for the applications running in the application pool process space. The various application demands usually drive the memory usage to over 800 megabytes. While the limit of the number of application pools is influenced by available memory, the general guideline for acceptable performance is to use ten or less.

 
 

You can have up to 5 zones per web application.

 
 

You should have less than 300 content databases per web application

 
 

You should have less than 50,000 site collections per content database

 
 

You should have less than 250,000 sites per site collection.

 
 

Host-named site collections are an option if you want to create multiple root-level site collections within a web application. You can create host-named site collections by using PowerShell. Additionally, by using PowerShell, you can use managed paths with host-named site collections. (New-SPManagedPath -HostHeader)

 
 

Host-named site collections give you more control over URLS. However, host-named site collections are only available through the default zone. User accounts that are configured to authenticate other zones cannot access host-named site collections.

 
 

In SharePoint 2010, host-named sites collections support off-box SSL termination for the protocol scheme. (http:// or https://)

 
 

You can create up to 100,000 host-named site collections within a single IIS Web site.

 

 

You would like to keep your _Total processor time between 50% – 75%.

 
 

Your processor queue length should kept below two times the number of core CPUs

 
 

Memory actually has two key areas: Available Memory and Pages per second. You need to target keeping 20% of the total physical RAM free while keeping the pages per sec below 100.

The most common cause of poor performance in earlier releases of SharePoint Server is the development and deployment of inefficient custom features on top of the SharePoint platform. When developing customer features for SharePoint, there are a number of performance metrics you should monitor. These include, but are not limited to:

 
 

  • SQL Server round trips   For core pages, we recommend no more than 2-3 SQL round trips. Excessive round trips have the following deleterious effect on performance:
    • Increased end user response time due to greater server-side processing time
    • Reduced overall system throughput due to additional load on the database server.
  • SQL server CPU utilization   In order for your MOSS system to remain healthy, it is important that CPU utilization on the database server(s) remains relatively low. If SQL Server 2005 CPU usage averages more than 60%, performance will be adversely affected. Steps you can take to reduce SQL CPU utilization include:
    • Implement a caching strategy – this reduces the overall number of calls from the Web server(s) to the database server.
    • Optimize custom code to use object methods which return your desired data in the most efficient manner (e.g. introduce indexes on lists, etc.)
    • Distribute your SQL databases across multiple physical database servers
  • Page download size   Keep code size to a minimum. A relatively small increase in page size can have a significant impact on performance if that page is accessed by a large number of people every day, especially during peak hours.
  • Client-side code efficiency   Approximately 50% of end user response time is comprised of client side processing of returned code. If your custom solution increases any of these, you can expect an adverse effect on end user response time.
  • AJAX callbacks   For AJAX parts, the number of callbacks, and the payload for each callback. For example, each KPI makes 3 calls in order to return the result. Make sure you test page performance when you introduce multiple KPIs or other custom code into a page.

 
 

You should design your code or performance using best practices and try to re-use existing client code versus adding more. As with any custom code, you should profile your solution so that you understand the potential limitations.

Microsoft SharePoint Server 2010 provides three types of caches that help improve the speed at which web pages load in the browser: The BLOB cache, the page output cache, and the object cache. The BLOB cache is enabled and configured in the Web.config file in the web application to which you want to apply the cache. The changes you make to the Web.Config file will be applied to all of the site collections and sites within the Web application. The page output cache and object cache are usually configured in the user interface at the site collection level; however certain settings for these caches can be configured at the Web application level.

The BLOB cache is stored directly on the hard disk drive of the front-end Web server. The first time that a page is called, these files are copied from the database to the cache on the server hard disk drive, and all subsequent requests for those files are then served from the hard disk drive cache of the server. By default, the BLOB cache is off and must be enabled.

 
 

We will take a look at how to configure this in our upcoming demo.

The page output cache stores the rendered output of a page. It also stores different versions of the cached page, based off of permission of the users who are requesting the page. Page output cache settings can be configured at the site collection level, at the site level, and for page layouts. By default, the page output cache is turned off.

 
 

The page output cache uses cache profiles that specify how long items should be held in the cache. You can specify different cache profiles to be used for anonymous and authenticated users, which optimizes the use of the cache based on the authentication methods that are allowed on the site.

 
 

You can configure cache profiles settings for a Web application by editing the web.config file on the application server. The cache profile settings that you configure at the Web application level will be used for all cache profiles in the site collections for that Web application.

The object cache reduces the amount of traffic between the Web server and the SQL database by storing object – such as lists and libraries , site settings, and page layouts – in memory on the WFE server. As a result, the pages that require these items are able to be rendered quickly, increasing the speed with which pages are delivered to the client browser. Object cache settings can be configured at the Web application level, and at the site collection level. By default, the object cache is on at the site collection level.

 
 

You can optimize the object cache for a web application by specifying the size of the object cache. Specifying a larger number can enhance performance for some large sites at the cost of memory on each front-end Web server. You can configure other settings for the object cache at the site collection level.

 
 

To see an example of how this works, let’s take a look at the SharePoint request cycle.

 

For more detail on SharePoint and the HTTP request, check out the following blog article: http://shannonbray.wordpress.com/2010/04/04/in-the-beginning-sharepoint-2010-and-the-http-request/

 
 

 
 

If your sites use lots of bandwidth, or if you want to use bandwidth more effectively, enable compression to provide faster transmission times between IIS and compression-enabled browsers. If your network bandwidth is restricted, as it is, for example, with mobile phones, compression can improve performance.

IIS provides the following compression options:

  • Static files only
  • Dynamic application responses only
  • Both static files and dynamic application responses

 
 

Compression of dynamic application responses can affect CPU resources because IIS does not cache compressed versions of dynamic output. If compression is enabled for dynamic responses and IIS receives a request for a file that contains dynamic content, the response that IIS sends is compressed every time it is requested. Because dynamic compression consumes significant CPU time and memory resources, use it only on servers that have slow network connections but that have CPU time to spare.

 
 

Unlike dynamic responses, compressed static responses can be cached without degrading CPU resources

 

 

About these ads
  1. June 24, 2010 at 4:19 am | #1

    GREAT post!
    Thanks a lot for sharing!

  2. sqlguy
    October 19, 2010 at 11:54 pm | #2

    Hi Shannon,

    I’ve spent several days searching and com eup short on how to configure SP2010 so I can set a limit on the logging ( Usage and Health Data ) database so it wont exceed a set size and it also deletes data older than a certain age please?

    I’ve looked setting size limits for diagnostic logs, but these arent the same thing. If I set a physical size limit in SQL itself, I’m worried that it might crash the database. My thinkingis that Sp2010 itself needs to monitor and control the database size through deletion of older data.

    Any thoughts welcome.

    Thanks in advance,

    Cheers

    Steve

    • October 21, 2010 at 12:03 am | #3

      Hello Steve,

      The best way to limit the size is to restrict the days for the database. I believe by default that the data is restricted to 14 days with a max of 30. This is the settings available for configuration. I would discourage any other modifications or restrictions.

      I am guessing you want to restrict based on limited disk space. This is a very helpful database and it is one of the first ones that is suggested for offloading onto it’s own space. This database can become fairly large.

      I’m probably not much help here. I personally like having as much data as possible as there are some great queries to execute against this data.

  1. December 2, 2011 at 1:18 pm | #1

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 29 other followers

%d bloggers like this: