案例研究

Cornerstone OnDemand Sees Game-changing Innovation with Cisco UCS and Fusion ioMemory Solutions from SanDisk

Cornerstone OnDemand’s Software-as-a-Service solution enables companies to recruit, train, empower, and manage their people. By deploying Cisco UCS Rack Servers with Fusion ioMemory PCIe application accelerators the Company has been able to reduce database response time, increase availability, and offer better and more innovative services to their customers.

Solution Focus

  • Software-as-a-Service
  • Microsoft SQL Server 2014
  • Information on Demand
  • Quality of Service

Summary of Benefits

  • 40% improvement in database response time
  • 100X reduction in database server failover time
  • Ability to provide near real-time data warehouse

Products

  • Fusion ioMemory SX300 6.4TB PCIe application accelerators
  • Cisco UCS C460 M4 Rack Servers

Summary

Cornerstone OnDemand’s Software-as-a-Service solution enables companies to recruit, train, empower, and manage their people. By deploying Cisco UCS Rack Servers with Fusion ioMemory PCIe application accelerators the Company has been able to reduce database response time, increase availability, and offer better and more innovative services to their customers.

 

Summary of Results:

  • General 40% improvement across the board on all production IIS traffic
  • Reduction in database server failover from 10 minute outage to a seamless seven second “hiccup”
  • Ability to provide a near real-time data warehouse solution to clients
  • Migration completed ahead of schedule due to speed of Fusion ioMemory cards and UCS efficiency
  • Increased operations efficiency through visibility of systems status provided by UCS
  • New, diverse storage environment allows for deeper troubleshooting and resolution of issues
  • Increased headroom in available capacity allows for additional services without additional hardware investments
"We are growing into the cards right now. The low latency on the cards did provide a reduction on CPU usage on our servers. That has also enabled us to start new projects for better service to our clients."

Reza Seraji,
IT Architect, Cornerstone OnDemand

Background

Cornerstone OnDemand helps some of the world’s largest organizations to recruit, train, and manage their respective workforces. The Company’s software has an impact on every aspect of the employee experience through workforce engagement and empowerment, which ultimately translates into greater business results.

Since 1999, Cornerstone OnDemand (CSOD) has been an innovator in cloud technology, delivering a Software-as-a-Service solution to ensure that clients are always on the latest version of Cornerstone, reducing IT and maintenance costs, and enabling integration with other technology platforms. Users can conveniently access CSOD software anytime, anywhere, and on any device.

Cornerstone OnDemand boasts as many as 24 million users associated with more than 2,600 corporations across the globe. Client companies may have as few as 5,000 users or as many as 250,000 users. “In some cases, these clients use our software for compliance purposes to make sure their users have taken the training they are required on an ongoing basis,” said Reza Seraji, IT Architect, Cornerstone OnDemand. “For example, a life sciences organization may need to comply with user training and government regulations and may need to make sure all employees complete their training on time. It is a reportable process. Therefore, this is a very important piece of software for our clients.”

 

The Challenge

Cornerstone embarked on a journey to overcome some challenges that the IT team had been facing with Microsoft SQL Server clusters as well as taking the opportunity to improve their already redundant storage footprint. The team decided to upgrade to a new configuration that included Windows Server 2012 R2 with Microsoft SQL Server 2014 AlwaysOn running on Cisco UCS C460 M4 Rack Servers using some of the fastest storage technologies available today.

The previous architecture had employed traditional Windows Failover clustering running Microsoft SQL Server 2008 with active/passive nodes. There were multiple nodes in the Windows cluster and they would each run a certain instance of SQL Server. Some of the challenges that Cornerstone had experienced in the past had to do with vendor software glitches that created a single point of failure, despite the redundant controllers on the storage arrays.

Another challenge was due to having multiple types of systems in the Cornerstone IT environment. Depending on the age of the server, each server was on a different service contract, required a discrete inventory of spare parts, and had a unique and disparate method of out-of-band management access. Mismatches in firmware, together with these challenges, had resulted in complications and frustration for the IT team.

When moving to a new architecture, the team elected to continue to work with Cisco. “We had a strong relationship with Cisco and had a positive experience with the team and their ongoing support,” Seraji commented. “We chose Cisco hardware and haven’t had any problems and appreciate the centralized management that is available with UCS manager. We have had years of working together with Cisco and have had proven success.”

The Cornerstone IT team had begun deploying Cisco UCS servers four years prior. As more of their older servers began to go end of life, they replaced the older servers with UCS. “It has been a very smooth deployment, with very few issues. Overall it has been a great success. Since we have had such a good experience with UCS, we wanted to continue working with Cisco for our SQL Server 2014 project,” said Seraji.

In addition, the team wanted to have a diversified storage footprint—two different storage platforms—to be sure that they would not have any redundant points of failure in the system and thereby be able to overcome some of the issues they had seen in the past.

"Without this project we wouldn’t be able to provide a near realtime data warehouse solution to our clients. In the past, the data warehouse database was refreshed about four times per day. Now, we are able to use the secondary read-only database replicas to provide reporting capabilities where it would be only a 15-minute delay between changes on the OLTP database and on the replica reporting database. It has allowed us to provide a lot better service to our clients and new offerings to them just by virtue of the sheer speed of the storage solution and the availability of the technology it provides."

Reza Seraji,
IT Architect, Cornerstone OnDemand

The Solution

The Cornerstone team reached out to their Cisco partner team to determine what new technologies might be available to support the new deployment. In addition, they researched PCIe storage options as well as other storage technologies available and narrowed those alternatives down to four different vendors of PCIe storage cards. They began an on-site proof of concept process to evaluate product performance, explore the adoption rate in the industry, and determine how users liked using the new storage products through reference calls.

When selecting PCIe cards for storage purposes, the key criteria for the Cornerstone team included strong performance and reliability, along with high density and device capacity. “All of the PCIe cards on the market are quite performant,” Seraji said. “We were coming from providing 120,000 IOPS in our entire data center to these cards that could provide that and more on a per-card basis. However, capacity was definitely an important factor for us because we have so many customers—some of our clients are loading all of their historical HR and training data onto our platform. Because the Cisco UCS C-class servers can only take four storage cards, each PCIe card needed to be high capacity.”

Cornerstone chose Cisco UCS C460 M4 servers to take advantage of up to 25TB of usable PCIe flash storage, with the option of accommodating more than 50TB of total flash space. In turn, Cornerstone selected Fusion ioMemory PCIe cards from SanDisk based largely on the capacity of the cards. All the primary database files would now reside on Fusion ioMemory PCIe cards. “We left traditional Windows clustering where the storage layer is shared between all cluster nodes and disks would migrate between nodes when clusters failover and we went to a Microsoft SQL Server 2014 AlwaysOn high availability architecture using real-time database replication across unique storage devices.” explained Seraji. Each Cisco UCS server contained multiple Fusion ioMemory SX300 6.4TB PCIe application accelerators.

The IT team began testing the solution in the summer of 2015 in lower-tier environments. “We have three client-facing environments for our software— pilot, stage, and production environments,” Seraji told us. “Our pilot and staging environments do not have any SLAs attached to them, but they are still very important and have a high level of resiliency as they may be in use by the HR administrators at the companies that use our software. These users test configuration changes within their Cornerstone portal in those lower-tier environments. However, the production environment is where most of our traffic resides.”

In September 2015, Cornerstone began upgrading their servers, deploying the new servers with the primary Fusion ioMemory PCIe application accelerators, and secondary Pure Storage arrays. Due to positive testing results, flawless execution, and the speed of the Fusion ioMemory cards, the team was able to complete the migration by the end of October—much faster than anticipated and ahead of schedule.

The Cornerstone production environment is made up of numerous servers, each running multiple instances of Microsoft SQL Server 2014. There is a primary and multiple secondary instances amongst those nodes. The primary SQL instance always writes to the local Fusion ioMemory cards and the secondary SQL instance writes to Pure Storage LUNs. “What this provides is a diversified storage footprint using multiple unique storage technologies, such that if we did have a failure for some reason on the redundant storage arrays or the Fiber Channel network or something down the line, we could quickly make some changes to switch to asynchronous replication and just use the Fusion ioMemory cards,” said Seraji. However, the diversified storage footprint and use of Fusion ioMemory cards for primary storage meant that Cornerstone was no longer dependent upon shared storage. “This has allowed us to further troubleshoot and dig deeper into issues when they rarely occur versus focusing our efforts on quickly restoring service to our clients. This really goes a long way in finding the root cause of issues and trying to troubleshoot the issue while they are occurring,” said Seraji.

The Cornerstone environment is shuttered out by swimlanes. Each swimlane has its own database servers, application servers, and web servers, with a specified number of clients assigned to that swimlane. This is replicated multiple times in the US and UK data centers to provide smaller failure domains. Therefore if we do have an unexpected issue, the entire customer base is not affected and only one smaller failure pod will be affected.

"We were coming from providing 120,000 IOPS in our entire data center to these cards that could provide that and more on a per-card basis. However, capacity was definitely an important factor for us because we have so many customers—some of our clients are loading all of their historical HR and training data onto our platform."

Reza Seraji,
IT Architect, Cornerstone OnDemand

The Result

The team has seen a number of benefits now that all the instances of SQL Server 2014 are up and running. “We can use the primary instance for read/write capabilities and the secondary instance for read-only capabilities, such as data warehousing. This SQL Server 2014 project has really been a game-changer for us,” Seraji confirmed.

The team has also detected improvements in application performance since the deployment, such as lower latency and reduced average page response times. Not only have customers commented on application performance improvements, but also the Cornerstone CTO has been pleased with the project and the diversified storage footprint.

“When we monitor our servers, one metric we watch closely is IIS latency, which is the total round-trip time of the request which begins when the client clicks on the web site, goes through the firewall, then passes through our web and application servers, to our database server, and finally ends when data is returned back to the client,” Seraji clarified. “We saw a general 40% improvement across the board on all our production IIS traffic.

The project has also opened the door for the IT team to be innovative in the types of solutions they can provide to Cornerstone clients. “Without this project we wouldn’t be able to provide a near real-time data warehouse solution to our clients. In the past, the data warehouse database was refreshed about four times per day. Now, we are able to use the secondary read-only database replicas to provide reporting capabilities where it would be only a 15-minute delay between changes on the OLTP database and on the replica reporting database. It has allowed us to provide a lot better service to our clients and new offerings to them just by virtue of the sheer speed of the storage solution and the availability of the technology it provides.”

Since SQL Server 2014 AlwaysOn provides a secondary read-only copy, the team utilizes that hardware for the reporting side. Since the storage had been so much faster than the previous storage platform, they can use it a lot more efficiently and can have more things running in parallel, whereas in the past they were constrained by IOPS. “For example, our previous storage array was able to provide about 120,000 IOPS at peak for the entire data center. Now, the Fusion ioMemory cards do multiples of that per card. The Pure Storage arrays we use, as secondary storage are making continuous improvements to get to that 200,000 to 300,000 IOPS mark as well,” said Seraji.

One of the biggest benefits is the centralized management and visibility. “When you look in UCS manager, you can count on having the visibility into every single device and every single level of firmware. With UCS manager, it is easy to create policies and ensure that everyone adheres to them. It makes management and upgrading much easier than having to go into the individual servers to make updates.”

In addition, while cluster failovers in the Cornerstone environment used to take 10 minutes, failovers now take a fraction of that time. Previously, SQL Server would shut down and start back up on the other node, caching all the procedures. “It could take some time for the server to start up and cache all the execution plans—especially because we are a company with so many customers and so many dedicated databases,” said Seraji. “It used to be a really impactful event when we did a cluster failover since it could be about a ten minute outage to our clients.” However, with SQL Server 2014 AlwaysOn, the failover is relatively seamless, since both instances are ready and doing high availability replication in the background. “For our application, it is now about a seven second hiccup when we fail over.”

準備好快閃向前了嗎?

無論貴公司是《財星》雜誌排名前 500 大企業或五人小型創業公司,SanDisk 都有能助您將基礎架構發揮最大功能的解決方案。

透過
電子郵件

請不吝提問,我們會盡快回覆。

與我們談談
800.578.6007

別再猶豫,立即與我們聯繫,開始建立完美的快閃解決方案。

業務洽詢

無論您是想先提出幾個問題,或是已準備好討論符合貴組織需求的 SanDisk 解決方案,SanDisk 銷售團都很樂於隨時提供服務。

請填寫下列表格,我們很榮幸能回答您的疑問,並展開討論。若您需要直接與銷售團隊討論,請來電:800.578.6007

欄位不可為空白。
欄位不可為空白。
請輸入有效的電子郵件地址。
欄位中只能包含數字。
欄位不可為空白。
欄位不可為空白。
欄位不可為空白。
欄位不可為空白。

請指出您有興趣的領域:

提問或意見:

您必須選擇一項。

感謝您。我們已收到您的要求。