This blog is about another common error reported by many DBA to me. He faced error related to create file in OS.

Here is the text of the message.

CREATE FILE encountered operating system error 5(Access is denied.) while attempting to open or create the physical file ‘C:\Program Files\Microsoft SQL Server\MSSQL14.SQL2017\MSSQL\DATA\DetachTest.mdf’. (Microsoft SQL Server, Error: 5123)

Important piece in above error message “operating system error 5(Access is denied.)”. From that it is clear that its due to permission issue with the file.

Why this happens?

Whenever a database is detached, the permission of the database files is modified. If any SQL Login has detached the database, then permissions are given to SQL Server Service account. If any Windows account has detached the database, then the permission is given to the account who has detached itself.

To explain the real cause, I decided to reproduce the error. I created a database called DetachTest.  A SQL Service account was set to CONTOSO\SQLService. Now, I logged into the server via SSMS with login CONTOSO\syadmin.

Then I detached the database and here are the permissions. You can view them by going to the file > properties and security tab.

Now, I logged in as “sa” account, which is a SQL Login. As I explained earlier, SQL Server would try to use a service account to read the file. As you can expect, the file is not readable by any account other than CONTOSO\sysadmin.

WORKAROUND/SOLUTION

By reading the blog so far, you must have got a clue about what needs to be done to avoid “Access is denied” error. If not, then here are the steps.

• Right-click on the file which is mentioned in the error message and go to properties.
• In the Properties dialog box, go to Security tab.
• In the Select Group or user names box (which has sysadmin in above image). Click the Edit button and enter the name you used to login to SQL Server. If its windows account, then use that account else use service account.
• Click Check Names and verify that the name you are specifying does exist as SQL Server login.
• Click OK.
• In the Group or User Names box, select the user name you selected in the above step and give full control permission to the user. Click OK.

Now you are all set to attach database files by using the SQL Server authentication or Windows authentication.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – FIX: Msg 5123, Level 16 – CREATE FILE Encountered Operating System Error 5

I always try to keep myself up-to-date with new released of SQL Server. This blog is part of my learning, while using SQL Server 2017, which I always share with my blog readers. This is the error related to Error 21 about a device not ready.

Note: I am using the pre – release version of SQL Server 2017, so information might change in future.

I was looking at ERRORLOG of SQL Server 2017 instance and found below the message, which I have not seen earlier.

2017-09-19 01:28:28.490 spid52 The operating system returned the error ’21(The device is not ready.)’ while attempting ‘GetDiskFreeSpace’ on ‘A:\’.

The error was clear that it was trying to get information from A drive on this machine (which is a floppy drive)

While looking further, I figured out that this was caused due to new DMV

Later I realized that it happened when I was using Restore Database in SSMS and looks like that time the DMV was fired. I was able to confirm that using profiler and launching the UI again.

To double check, I also ran the same DMV manually and I can see that again but not as an error message but as an informational message.

To reproduce, you can run below command on SQL Server 2017.

Select * from sys.dm_os_enumerate_fixed_drives

If your machine has a DVD drive, then you may also get

2017-09-20 14:47:24.380 spid58 The operating system returned the error ’21(The device is not ready.)’ while attempting ‘GetDiskFreeSpace’ on ‘D:\’.

The only difference is the drive letter.

Based on above, I think we can safely ignore this error message in the SQL Server ERRORLOG.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – The Operating System Returned the Error 21(The Device is Not Ready.) While Attempting ‘GetDiskFreeSpace’ on ‘A:\’

A couple of months ago I told you about the new interesting tool I found for documenting existing database schemas – Dataedo. For those who don’t remember, this is a desktop tool that enables you to describe database schema elements and create ER diagrams for existing databases, and export to nicely formatted HTML and PDF documents.

Since my last article I have worked with the tool more, and, with the courtesy of the guys from Dataedo team, I had a sneak peek into their upcoming new major release – Dataedo 6. I believe it now stands out even more and, despite being simple, is ready for serious enterprise data governance programs. Let me walk you through the features.

Why document databases?

Let’s have a look first why you should use a database documentation tool. Some of you might think that the database schema is self-documented, that all you need is a database console and you are ready to query the data. Well, not exactly. Here are some reasons why that’s not that simple:

1. The database is missing FK constraints and table relations are unknown,
2. Tables and columns have confusing names,
3. There are hundreds or thousands of tables and it’s hard to find the data,
4. There are a lot of unused tables and columns and there’s a risk of using obsolete data,
5. It’s not clear what data tables and columns actually hold.

With up to date documentation developers, report creators, data analysts, and organizations as a whole can:

1. Make better use of their data,
2. Make it easier to replace and onboard new people,
3. Create more reliable analytics and reporting,
4. Save time on database discovery and learning,
5. Maintain legacy databases.

Data has not much value if you don’t understand it or can’t find it.

Works for enterprises and small projects

What I like about the tool is that you can generate good looking documentation of your databases in a couple of minutes. It is fairly simple and small tool. And yet, with features like documentation repository, teamworking, advanced metadata capabilities, diagrams, etc. it is ready for some serious enterprise applications.

Cross database diagrams

A notable feature of the new release, something I have seen only in advanced modeling tools (which might be too heavy for documenting database schema, and definitely not user-friendly for describing data dictionaries and clunky exports) is the ability to document models that span many different databases, servers or even platforms (i.e. Oracle, SQL server, MySQL).

You can define a link between any two tables, no matter to which database they belong, and mix those tables in the diagrams.

This gives enables documenting complex environments.

Data Dictionary

This is one of a key features of the tool – easy way and publish a Data Dictionary (a definition and description of all the tables and columns). You can document not only tables, but many other database objects.

Full documentation – database schema, descriptions and diagrams are exported to a nice HTML document.

Keeping model up to date

One of the issues when managing documentation outside of the database is that when schema changes it is really hard to keep the documentation up to date and in sync with the database. If you’d like to do that in Excel it could be a nightmare.

Dataedo handles this nicely. At first, you import schema to new “documentation” and then you can always automatically update the schema of your documentation from the live database. New elements are added to the data dictionary and removed are marked as deleted.

Extensions with custom fields

Another very interesting and powerful new feature of Dataedo 6 that makes a real difference is the ability to define multiple custom descriptive fields. You can set up a set of attributes you want to add to columns, tables and other database elements that you can easily access, edit and export. This is way more convenient than adding Extended Properties with SSMS in the SQL Server.

Publish documentation online

Dataedo enables exporting documentation to printable PDF, Excel that you can use as a data source or HTML that you can publish online, in your intranet or simply save to the local or network disk. It is also possible to publish it on portals like SharePoint or Confluence.

I tried it on Confluence and it looks quite good.

Keep documentation in repository and database

Dataedo 6 keeps all metadata in repository but also enables you to write them back to the database into extended properties (in case of SQL Server) or comments (in case of Oracle or MySQL). You have the best of two worlds:

1. Advanced description capabilities, dependence on, no need for access to live database or alter its schema, the benefits of repositories and
2. Documentation with your database and data, available for use by other tools.

Open repository & interface

Dataedo repository is in fact a plain SQL Server database that you can query. The online documentation has a few useful scripts you can run against the repository to modify metadata. This gives abilities to automate documentation and integrate with other processes.

Working in development environment (dev and prod environments)

Another advanced capability is the ability to set up repositories for different environments – development, UAT and production – and release changes so you can incorporate documentation in your development process.

Process:

1. Developers apply changes to dev database schema
2. Scheduled Dataedo process imports schema changes from dev database to dev documentation
3. Developers update documentation (describe new objects)
4. Scheduled Dataedo process exports dev documentation and shares with the dev team
5. Changes are being deployed to prod database (dev tools)
6. Scheduled Dataedo process imports schema changes from prod database to prod documentation
7. SQL script moves dev documentation to prod repository
8. Scheduled Dataedo process exports prod documentation and shares with the organization

Automation

You can easily schedule tasks and create an automated process of documentation generation and publishing so that all the developers, DBAs, and other database users have access to the most recent documentation.

The future

Looking at the website and talking to the team it seems that the tool is heavily developed and there’s a lot of other interesting features on the way (or at least explored). Including more database objects and engines being one of the things developed. I think it’s worth having an eye on this tool.

Conclusion

I already liked the tool in its previous version and the new release makes it even more useful and advanced. If you are not documenting your database schemas already, It’s a great opportunity to change that!

Download Dataedo

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Document Your Databases with Data Dictionary and Diagrams

As a part of my relationship with existing clients, I provide quick help to them on simple issues. Sometimes it is an interesting issue and I write a blog about it as well. Let us learn the error related to SaveToSQLServer.

John contacted me and told that he is not able to save the maintenance plan after Selecting All Databases while create a backup plan using maintenance plan wizard. Here was the error:

The SaveToSQLServer method has encountered OLE DB error code 0x80040E4D (Login failed for user ‘sa’.). The SQL statement that was issued has failed.

I searched for SaveToSQLServer but found nothing interesting. Then I looked at the second part of the error and we can see “Login failed for user ‘sa’”. When I asked about it, John informed me that it works after 10 minutes and we should use windows authentication. Interestingly, when we used windows authentication, it was working fine.

I took a step back and told John to explain this behavior and he was not aware of the cause of this auto login failure.

WORKAROUND/SOLUTION

We checked ERRORLOG and found that login for ‘sa’ was failing because it was getting locked out. We looked into login properties and found that the ‘sa’ login has enforce password policy enabled. Now next task was to find out why the account was getting locked.

We used profiler and found that there was another maintenance plan which was connected with an old ‘sa’ password and causing login failures. Due to password policy, it was getting locked out. The domain was set to unlock password after 10 minutes, which explained why it was working for some time.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – The SaveToSQLServer Method has Encountered OLE DB Error Code 0x80040E4D

In today’s blog post we are going to learn how to connect specific database directly when we start SQL Server Management Studio. Honestly, this is a very simple process, but I have seen quite a few DBA still unaware of this feature.

I have seen DBAs directly connecting to master database, which has eventually lead them to create objects which does not belong to master database. If you do not believe me, you should open your master database and see that there may be quite a few objects which are not related to that database at all.

Let us learn how we can configure our SQL Server Management Studio (SSMS) to connect to specific database when we start it off.

First open SSMS Connect to Server Screen –

Next Select Options >>

Right after that click on the drop down “Connect to Database“

Over here – click on “Browse Server…” and next it will bring another popup where it will say, that it may take some moments. Click Yes over here.

Finally, it will bring up the Browse Server for Database Screen.

Select your preferred database and click on OK.

That’s it. You are done. Now you will be connected to your preferred database when you connect to SQL Server Management Studio (SSMS).

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Connecting Specific Database on Starting SSMS

The database world is very big and it is very difficult to compare multiple databases with each other. The reality is that every database organization talks about the unique features of their database in such fashion that each database is the best among the competition. When I go for a database consultation, my customers often ask, which is the best possible database out there in the industry which can handle different operational workloads efficiently. If you search on the internet, you will realize that there are tons of the database products out there. In this blog post we will discuss about how we can select the best possible database with the help of YCSB Benchmark.

Before we continue further, let us understand what is YCSB Benchmark.

YCSB Benchmark

As per Wikipedia – The Yahoo! Cloud Serving Benchmark (YCSB) is an open-source specification and program suite for evaluating retrieval and maintenance capabilities of computer programs. It is often used to compare relative performance of NoSQL database management systems.

So in the simple words – we can use this benchmark to compare multiple open source databases and find out which one is the most efficient database. This benchmark was originally developed by the research division of Yahoo! in the year 2010.

The best part of the YCSB benchmark is the extensibility, it can be easily extended to support new workloads for absolutely new system.

The best part of the YCSB benchmark is the extensibility, it can be easily extended to support new workloads for absolutely new system.

Here are three major characteristics of the cloud serving system.

• Scale-out
• Elasticity
• High Availability

This particular benchmark focuses the various available tradeoffs in the cloud system. Here are the major tradeoffs for the cloud serving systems.

• Read performance versus write performance
• Latency versus durability
• Synchronous versus asynchronous replication
• Data partitioning

Additionally, the benchmark process is done in various different tiers.

• Tier 1—Performance
• Tier 2—Scaling
• Tier 3—Availability
• Tier 4—Replication

We can benchmark new database system via writing new classes for read, insert, update, delete and scan methods. I, personally, believe this particular behavior of the benchmark has made it extremely popular in the big database industry.

Recent Benchmarking Exercise

Recently solution architecture from the NuoDB ran a benchmarking exercise for Google Cloud Spanner, CockroachDB and NuoDB. They ran the following workload which are standard in the YCSB benchmarks –

• A – Heavy update (50% read, 50% update)
• B – Mostly read (95% read, 5% update)
• C – Read-only (100% read)
• D – Read the latest inserted (90% read, 10% insert)
• E – Scan the latest inserted (90% read, 10% insert)
• F – Read-modify-write (50% read, 50% update)

Here are three major graphs from the YCSB Benchmarking.

This graph measures the number of transactions per second for each database during peak throughput. In all five of the tested workloads, NuoDB significantly outperforms both Cloud Spanner and CockroachDB.

This graph measures average latency experienced by the application for READs during periods of peak throughput. Minimal latency is ideal for the best user experience.

This graph measures average latency experienced by the application for UPDATEs and INSERTs during periods of peak throughput. Minimal latency is ideal for the best user experience. Note that Workload C is not represented here as it is a READ-only workload.

Conclusion

You can clearly see that NuoDB is flexible and fast. It can achieve superior performance for throughput and volume. You can read about this entire benchmarking exercise over here: Benchmarking Google Cloud Spanner, CockroachDB, and NuoDBND.

I suggest you download NuoDB and try it out yourself.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on YCSB Benchmark and Latest Evaluation With NuoDB

I just love my On Demand offering because it gives me more pleasure of assisting clients who are in urgent need of an SQL expert. Most of the time I am called for help when clients are not able to start SQL Service due to some reason. Today we will see how to fix an error which is about unable to start SQL Server.

My approach to solve the SQL startup issue is straightforward. Look for error messages, wherever it could be. Mostly it is in ERRORLOG or Event Log.

Here is the last part of the ERRORLOG file.

As soon as I saw “The system cannot find the file specified.” message, I started Process Monitor and capture data and compare with my laptop. It took a lot of time to identify the issue, but it was fun. I checked “SuperSocketNetLib” key for the instance.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL13.MSSQLSERVER\MSSQLServer\SuperSocketNetLib

When I compared above with my laptop, I found that Certificate key is missing.

SOLUTION / WORKAROUND

I created a Certificate key as shown on my workstation and I could fix the issue. Then we were able to start SQL Server.

When I asked them about the key they informed that they were earlier using certificates. Since they are going away with it, someone advised them to delete the certificate thumbprint from registry also and they misunderstood.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Unable to Start SQL Server – TDSSNIClient Initialization Failed with Error 0x2, Status Code 0x38

With the new Intel Skylake CPUs coming out recently it seemed like a good time to discuss CPU resources and how to know if they’re running optimally.  After all, there’s little sense in upgrading your server if your current one is already data starved. Let us learn about how to balance your SQL Server facing data starvation.

What is Data Starvation? 

Data Starvation is when CPU cores have unused time slices due to waiting for data.  Databases need data and if time slices are unused, then it makes queries last longer, wastes CPU licenses, etc.  This is typically caused by an imbalance between CPU and I/O subsystem performance.

Skylake offers up to 26 physical cores per CPU.  With Hyperthreading, that’s 208 logical cores in a 4-CPU server (common for analytics).  That’s a lot of parallel streams to feed.

Back to Data Starvation.  There are two types:

• Bandwidth: Not supplying enough bulk data to feed the processors grinding through reports in parallel.
• Latency: Not returning I/O requests fast enough to keep the CPU cores fully active.

Bandwidth:

SQL Server has made I/O engine improvements over the versions (like larger read-ahead batch sizes, etc).  Multiply these by expanding the numbers of cores per server and that means vast amounts of data can be processed.

Ok, so what can a SQL Server do?

Recently I borrowed a SAN array from my friends at Vexata and ran some TPC-H like workloads.  I was able to get a single 2-CPU SQL Server running at 25GB/s of reads plus 11GB/s of writes for a total of 36GB/s.  That’s a ton of data crunching in one small server.

To create a ballpark method for sizing servers I divided my peak bandwidths by the number of cores on my two test systems.  This gives the per-core bandwidths the SQL Server is capable of.

Latency:

When SQL Server attempts a transaction the first step is to make I/O reads for the index and then data pages.  Transactions can’t start until the data has been loaded.  For larger tables, there can be 10 or more index levels to hop before getting to the data page.  And, this is done sequentially.  This is why common OLTP workload testing profiles are 70/30 r/w as the most common I/O is the initial index traversing and data reads.  If applications seem sluggish and the cores aren’t over 90% then I/Os could be pending.

Bad: CPU Starved

Good: CPU Optimized

In part 2 we’ll cover how to use SQL Server tools to determine if data starvation is your bottleneck.

Call to Action: Meanwhile, check out Vexata solutions for SQL Server.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on Data Starvation – Balance Your SQL Server – Part 1

Along with my freelance work as an independent consultant, I do keep an eye on various forums and look at issue reported with solutions so that I can blog about them to help the community. One of the Facebook group where I am active is SQLBangalore and recently once of the member posted an issue. He was trying to create a login using SQL Server Management Studio and getting error for SELECT INTO statement as below.

Here is the text of the error message.

An object or column name is missing or empty. For SELECT INTO statements, verify each column has a name. For other statements, look for empty alias names. Aliases defined as “” or [] are not allowed. Change the alias to a valid name. (Microsoft SQL Server, Error: 1038)

I was interested to know which command is raising the error, so I captured profiler and found below.

CREATE LOGIN [] FROM WINDOWS WITH DEFAULT_DATABASE=[master]

If we run above in the SSMS query window, we get below

Msg 1038, Level 15, State 4, Line 1
An object or column name is missing or empty. For SELECT INTO statements, verify each column has a name. For other statements, look for empty alias names. Aliases defined as “” or [] are not allowed. Change the alias to a valid name.

Now the error makes sense because login is created as [] and error clearly says that “…[] are not allowed…”

As explained by other members, it was due to the fact that login name was not given in the “general” tab. Anyone can reproduce the error by hitting OK on the login page without doing anything.

SOLUTION/WORKAROUND

In this situation, the issue was caused because of the empty login name. If we provide name, then the issue will be resolved.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Msg 1038 – An Object or Column Name is Missing or Empty. For SELECT INTO Statements, Verify Each Column Has a Name

While testing a scenario for my client, I ran into a behavior which I was not aware of. Let us learn about the error which is related to Empty Startup Parameters in SQL Server Configuration Manager.

I logged in to the virtual machine (my lab environment) and wanted to change ERRORLOG location. You can refer my below blog to find various ways to know location of ERRORLOG file.

SQL SERVER – Where is ERRORLOG? Various Ways to Find ERRORLOG Location

I used configuration manager and I was surprised to see that there was no parameter listed.

When I checked other system, there is no way to remove those parameters from the interface. I searched further and found that they are stored in the registry key. I opened below key.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQLServer\Parameters

As soon as I clicked on parameter key, I was welcomed by below message.

It was clear now that due to permission issue on registry key, configuration manager is showing empty box. To confirm this, I checked another working machine and found below permission.

As we can see, the permission is restricted to Administrators group on the machine, SYSTEM account and Service SID.

SOLUTION

To view startup parameter, we should log into the machine with an account which has Administrator privileges on the machine. In the above test, I was logged in with an account which was a normal user on the server. When I logged off and logged in with the Administrator account, I was able to see the startup parameters as expected.

Do you have such restrictions in your production environment?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Empty Startup Parameters in SQL Server Configuration Manager

Today, we are going to talk about one of the most popular errors which user see every day. There are many users who often get stuck at this error and have no clue how to move forward with this error in SSMS. It is quite possible you may say this is a very simple error and you knew about it, well good for you. Remember this blog post is for everyone who have faced error 208 – Invalid Object Name and have no idea how to move forward from this point.

First, let us see how this error happens.

When a new user is trying to explore any database, they usually, go via the route of SQL Server Management Studio. When they try to access any object via any valid T-SQL command, often they end up with the following error.

What primarily confuses users is that they are able to see that object is clearly visible in the SSMS Object Explorer, but when they try to access that via valid T-SQL they usually face the error. I have seen DBA and Developers to often restarting SSMS, restarting machine and furiously trying to refresh their queries and database helplessly. However, the problem is very simple one and almost all of us learn it eventually.

Workaround/Solution:

This error happens primarily due to database context mismatching.  When people open SSMS, they usually do not notice that their default database context is set to master in the most of the cases. We just have to change the database context to whatever database our current object is and everything will start working immediately.

This is so popular error that you will find every single DBA in the world experiencing it quite frequently. However, as the solution is so quick, we all learn how to fix it.

Another alternative would be prefix your database with a multipart database name. You can read about it over here: SQL SERVER – Explanation and Example Four Part Name

Additionally, you can also change the default database when you login in SSMS to another database than the master database. You can read about it over here: SQL SERVER – Connecting Specific Database on Starting SSMS

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Error Due to Database Context – Error 208 – Invalid Object Name

If you are reading this blog, you must be aware that I focus on relational database and also in particular for SQL Server Performance Tuning. I always believed that any system in the world can use more performance. Well, while I focused on relational database, I accept that I have focused very little on the subject of PowerBI and Advanced Analytics.

When users want to learn about relational database concepts, there are quite a few places one can learn about this topic. However, on the subject of PowerBI and Advanced Analytics there are very few quality resources. Well, today, I am going to answer you one question which I often get asked.

Where can we learn about PowerBI and Advanced Analytics?

If you are one who has ever wondered about this question, I think I have answer for you.

Self Paced Learning

Self-paced training videos, Learn from your desk at the office or home at your own pace.

Advanced Data Transformation and Preparation with Power Query in Power BI
In this course, you will learn all things about Power Query, from zero to hero. You will learn Power Query from a basic level with Getting data from different data sources. You will learn about different types of transformations available in Query Editor. You will also learn about M (Power Query Formula Language) in deep.

Power BI Essentials
This course designed to give you an end to end view of all Power BI components and tools. In this course, you will learn all modules of Power BI; from Getting Data with Power Query, Doing data Modeling and DAX, Visualization in Power BI Desktop, Publishing and Sharing content in the Power BI Service, and Refreshing Dataset and Security.

Instructor-Led Training

Power BI from Rookie to Rock Star – Live 5-days Course
Learn Power BI from Rookie to Rock Star; beginner to advance. Through extensive 5-days training course (lecture and lab). Learn from introduction of Power BI, to Get & Transform, Data Modeling, Visualization, and Deep dive into Power Query Formula Language, DAX, Custom Visualizations, Power BI Embedded, Real-time solution with Power BI, Gateways, and many more.

Advanced Analytics with Microsoft Technologies – Live 5-days Course
You keep hearing about the machine learning and R recently. R become a language for data analysis and machine learning that makes the need for learning it more clearly. Moreover, Microsoft introduces R server as a comprehensive platform for using machine learning and R functionality inside the data analysis tools such as SQL Server 2016 and Power BI.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on Learn PowerBI and Advanced Analytics – Instructor-Led and Self Paced Training

In Part 1 I covered how much data your SQL Server should be processing and the effects if it isn’t.  Here we check to make sure that you’re achieving that speed. Let us continue learning about data starvation in this blog post. We will be continuing to see how Vexata helps our SQL Server run faster.

Are the CPUs consistently over 90% utilized?  If not, then something is holding them back.  This could be application or hardware issues, below we’ll attempt to eliminate hardware.

First, look at the % User Time in Performance Monitor.  SQL Server runs in the user space and low %’s of time spent in the user space could signal the CPU is being held back.  Like in the below screen grab, simulated on a server connected to a slow SAN, only about 14% of the CPU’s 84% (% Processor Time) was actually spent on SQL Server work.

Verify server hardware:

Transport is the highway between your CPUs and your storage and should be more than big enough for your number of cores.  Per port speeds, one way:

• 10Gb NIC (iSCSI): ~550MB/s
• 8Gb HBA (Fibre Channel): ~800MB/s
• 16Gb HBA (Fibre Channel): ~1,600MB/s
• 32Gb HBA (Fibre Channel): ~3,200MB/s

Know your workloads:

Are your mission critical applications OLAP or OLTP?

OLAP:

Read and write bandwidth are the most important.  The higher the bandwidth the faster reports will go, the more can run at the same time and the faster the data will load.  Latency should still be low (<2ms), but not as low as the smaller batches in OLTP.

Test:  Run a table scan on a large table and see how high the bandwidth goes.  Limit the results returned with a filter like: “SELECT * FROM CUSTOMERS WHERE AGE < 0”.  Monitor the bandwidth with PerfMon.

Analysis:  The read bandwidth should sustain around your theoretical peak based on the number of logical cores as discussed in blog Part 1.  Note:  Using a table scan removes application based issues so full data throughput can be tested.

OLTP:

Read and write latencies are the most important.  The faster the CPUs can get back to processing the fewer time slices will be missed.  IOPs is a measure of scale.  How many operations can be executed.

Test:  Use PerfMon to record the CPU utilization and LogicalDisks’ IOPs and Latency for the data, log and tempdb volumes.

Analysis:  During peak user workloads, is the CPU utilization consistently over 90%?  If not, what are the latencies?  Lowering latencies get the CPU back to work faster, reduces wasted time slices and increases CPU utilization.

Common latency ranges for 8k OLTP IOPs running at full production load:

Conclusion

SQL licenses can be the most expensive part of a database platform.  If the CPUs are not consistently over 90% then investigate the situation.  A common issue these days is data starvation in which the storage tier is vastly underpowered and leaving the CPUs underutilized.

Call to Action: Meanwhile, check out Vexata solutions for SQL Server.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on Data Starvation – Balance Your SQL Server – Part 2

If someone tells me any situation, the first question which I always ask is “What is the error message?”. Now to get the right error message, it is important to look at the right log file. In this blog, we would learn about SQL Server Agent startup issue.

If SQL Server Agent service is not getting started, here are the logs I always look at:

1. out file. This is generally located in the same folder where we have SQL Server ERRORLOG file. You can refer to below blog to find the location. SQL SERVER – Where is ERRORLOG? Various Ways to Find ERRORLOG Location
2. If there is nothing interesting over there or the file itself is missing, then I would look at Event logs (both application and system). To open it, we can use the shortcut.

Start > Run > EventVwr.msc

For this client, here are the messages in various logs.

1. In SQLAgent.out, we found below messages.

2018-01-28 12:01:34 – ! [165] ODBC Error: 0, Driver’s SQLAllocHandle on SQL_HANDLE_ENV failed [SQLSTATE IM004]
2018-01-28 12:01:34 – ! [382] Logon to server ‘(local)’ failed (ConnConnectAndExecuteSQL)
2018-01-28 12:01:34 – ! [165] ODBC Error: 0, Driver’s SQLAllocHandle on SQL_HANDLE_ENV failed [SQLSTATE IM004]
2018-01-28 12:01:34 – ! [382] Logon to server ‘(local)’ failed (SendCryptoKeyToServer)
2018-01-28 12:01:34 – ! [442] ConnConnectAndSetCryptoForXpstar failed (0).
2018-01-28 12:01:34 – ! [165] ODBC Error: 0, Driver’s SQLAllocHandle on SQL_HANDLE_ENV failed [SQLSTATE IM004]
2018-01-28 12:01:34 – ! [382] Logon to server ‘(local)’ failed (DisableAgentXPs)
2018-01-28 12:01:34 – ? [098] SQLServerAgent terminated (normally)

From above, we could see some ODBC errors but not exact cause of the issue. So, as a next step, I opened event viewer and found below message at the same time when SQL Agent startup was attempted.

Log Name: Application
Source: SQLNCLI11.1
Event ID: 1
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: DBSRV1.SQLAuthority.com
Description: SQL Server Native Client 11.0: Unable to load sqlnclir11.rll due to either missing file or version mismatch. The application cannot continue.

WORKAROUND/SOLUTION

After seeing above mentioned message in application event log, I was convinced that there is some issue with SQL Server Native Client component. To fix this, we went to add/remove program and performed a repair of “SQL Server Native Client” as shown below.

Once we repaired the component, SQL Server Agent was able to start.

Have you encountered such issues which are solved by messages in the event log?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Unable to Start SQL Server Agent. Error – Driver’s SQLAllocHandle on SQL_HANDLE_ENV failed

Here is one of the situations which I encountered with SQL Server 2014 on Windows Server 2012 R2. I wanted to add “Analysis Services” features to an existing installation. I came up with the feature selection screen on SQL setup and pressed next it hangs with “Please Wait”. I click on it and went for lunch, when I came back, it was still in the same state. Let us learn how to fix the issue related to Installation Wizard Hang Forever.

Earlier, I have written blogs for setup hang situations which are below but they are not for the same problem.

SQL SERVER – Slow Installation Wizard on Cluster – Please Wait While Microsoft SQL Server 2016 Setup Process the Current Operation

SQL SERVER – Installation Wizard Hangs at Service Account Page – The RPC Server is Unavailable

I went ahead and looked into Detail.txt to see what is causing the hang.

Baseline Package Id: sql_engine_core_inst_Cpu64, ProductCode: {17531BCD-C627-46A2-9F1E-7CC920E0E94A}, The cached MSI file C:\Windows\Installer\1b04c4ec.msi is missing.
Error: Action “Microsoft.SqlServer.Configuration.MsiExtension.SetPatchInstallStateAction” threw an exception during execution.
Microsoft.SqlServer.Setup.Chainer.Workflow.ActionExecutionException: The cached MSI file ‘C:\Windows\Installer\1b04c4ec.msi’ is missing.
Its original file is ‘sql_engine_core_inst.msi’ and it was installed for product ‘SQL Server 2014 Database Engine Services’ from ‘R:\x64\setup\sql_engine_core_inst_msi\’, version ‘12.1.4100.1’, language ‘ENU’.

To resolve this problem, recover the missing file from the installation media and start setup again.

WORKAROUND/SOLUTION

If you read the information in Detail.txt, it is clear that I have deleted some files from C:\Windows\Installer folder which are needed for SQL Server installation process. This is my lab server and I sometimes do cleanup space. As a DBA, you should never delete files from C:\Windows\Installer folder.

To solved my problem, I used Microsoft’s knowledge base article.

Once I cached the missing files, I was able to move forward and add the feature.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Installation Wizard Hang Forever – Please Wait on Feature Selection Page

During one of my consultancy related to Always On availability group deployment, I faced an interesting issue. They wanted to install two standalone SQL Server instances on both nodes of the Windows Server 2008 R2 cluster for a specific reason. Interestingly, they informed that this used to work fine with SQL server 2008 R2 but in SQL server 2014 is failing with an error.
I checked the summary file and found the message about install error.

Overall summary:
Final result:                  Failed: see details below
Exit code (Decimal):           -2061893616
Exit facility code:            1306
Exit error code:               16
Exit message:               The directory J:\SQLSERVER\MSSQL12.MSSQLSERVER\MSSQL\DATA is not a valid path.
Standalone SQL Server on WFC cluster cannot have files on shared disk locations.
Start time:                    2018-02-04 08:51:28
End time:                      2018-02-04 08:52:58
Requested action:              Install

I searched on the internet and found that one way to achieve this is to first install standalone instances without the Windows cluster and then create windows cluster. In case cluster is already created then we can evict the node, install SQL Server on evicted node and then join the node back to Windows Cluster (WSFC)

Both of above methods would work but my client was looking for more options where they need not break the cluster etc.

WORKAROUND/SOLUTION

1. Here are the other workarounds which I provided where they can install SQL Server without breaking cluster.
   Rather than breaking cluster, we can stop cluster service on the node before installing standalone SQL Server instance. We just need to make sure that drive (J: in our example) is accessible from the node when service is stopped. This would give feeling to installation that this is a standalone node and it would work.
2. We can also try installing it via command prompt (I didn’t get chance to test this)

Setup.exe /q /ACTION=Install /FEATURES=SQL /INSTANCENAME=MSSQLSERVER /SQLSVCACCOUNT=”<DomainName\UserName>”
/SQLSVCPASSWORD=”<StrongPassword>”  /SQLSYSADMINACCOUNTS=”<DomainName\UserName>”
/AGTSVCACCOUNT=”NT AUTHORITY\Network Service” /IACCEPTSQLSERVERLICENSETERMS

Have you found any other solution? Please share with other readers via comments.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Install Error: Standalone SQL Server on WFC Cluster Cannot Have Files on Shared Disk Locations

While working with many clients, I learned that “there could be many problems leading to same symptoms”. In this blog, we would learn about a reason which can cause empty startup parameter in SQL Cluster.

Long ago, I worked with a client and found that registry permission can cause empty startup parameter in SQL Server Configuration Manager.

SQL SERVER – Empty Startup Parameters in SQL Server Configuration Manager

In the current client scenario, the account was already part of Local Administrators group so it was not due to permission. Interestingly, when I checked registry key, I found that there were no parameters there as well.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQLServer\Parameters

To make things more interesting, we found that on one node we were able to see values in the registry as well as in the SQL Server Configuration Manager.

WORKAROUND/SOLUTION

Based on my understanding of Windows cluster, it sounds that issue with registry key checkpointing. We found that there were some issues with cluster resources, so my client recreated them manually. Due to this the checkpoints were missing.

Here are the PowerShell commands to add registry checkpoint.

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\Cluster”

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\MSSQLServer”

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\Replication”

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\Providers”

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\SQLServerSCP”

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\CPE”

Add-ClusterCheckpoint -ResourceName “SQL Network Name (MSSQLSERVER)” -RegistryCheckpoint “SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL14.MSSQLSERVER\SQLServerAgent”

You should make changes based on your environment registry keys and resource name.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Empty Startup Parameter in SQL Cluster on One Node

After cleaning up some data from my user profile, when I launched SQL Server Management Studio 17.4, I got an error. related to automatically saved settings.

Here is the text of the message.

The automatically saved settings file ‘c:\users\<UserName>\documents\visual studio 2015\settings\sql server management studio\NewSettings.vssettings’ cannot be found. You can change this file on the ‘Import and Export Settings’ Tools Options page. The IDE will use your most recent settings for this session.

Once I clicked OK, I was able to get SSMS working and error never appeared. I was curious to know the cause of the error.  After capturing Process Monitor I was able to find the location from where SSMS picks the value of the settings file.

Most of the settings are stored in

HKEY_CURRENT_USER\SOFTWARE\Microsoft\SQL Server Management Studio\14.0

I noticed that I get this error first time when I rename “sql server management studio” folder to “sql server management studio_1” in windows explorer. When I opened SSMS, the folder got created automatically and hence no error appeared again.

Sometimes, you may get an error about network location so you need to find the key which is pointing to a network location and fix that.

Have you ever faced this issue? If yes, please share your experience.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – The Automatically Saved Settings file NewSettings.vssettings Cannot be Found

Some error messages are so interesting that it takes a lot of time to get the right solution. One of the errors which I faced during SQL Cluster installation is so common that my search was showing various solutions which were not applicable to me. After spending a lot of time, I was able to find the real cause. In this blog, we would learn about SQL Cluster installation error “The folder path ‘Program Files’ contains an invalid character “

THE PROBLEM

My client was installing SQL Server in the cluster. One first node they created a new failover clustered instance. On the second node, they were performing Add Node operation and it was failing with an error.

Detailed results:
Feature: Database Engine Services
Status: Failed: see logs for details
MSI status: Failed: see details below
MSI error code: 0x2AFCCCD0
MSI log file location: C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\Log\20180220_114356\sql_as_Cpu64_1.log
MSI error description: The folder path ‘Program Files’ contains an invalid character.

If we look at the file name it is “sql_as” which means this is for analysis services. I looked at the log mentioned above and found that setup was looking for D drive on the new node. This was due to the fact that Program Files were installed on D drive on the first node.

THE SOLUTION

It looked like this issue occurs if we have different drive letter names on both nodes of the cluster. For example, we install SQL Server binaries on D drive on one node. However, there is no D drive on the passive node.

• Uninstalled ALL SQL components that had been installed on passive node via the control
• Change the local drive letter on the passive node so that it matches to the local drive on the active node which contains SQL server binary files.
• Reboot the passive node and then re-run the setup.

This resolved the issue for my client. Have you encountered the similar error? What was the solution you opted for?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Error 1324. The folder path ‘Program Files’ contains an invalid character.

One of my blog readers contacted me via email for an error message which he was receiving while using the backup to URL feature. Using this feature, SQL Server can take database backup directly to Azure Blob Storage. I have already written a blog about few errors which I received earlier but this error was new. I replied him asking more details and it was an easy error to fix. In this blog we would discuss how to fix 3271, The remote server returned an error: (403) Forbidden.

Here is the command which I have used to reproduce the error.

BACKUP DATABASE SQLAuthority 
TO URL='https://sqlprodbkups.blob.core.windows.net/sqldbbackups/SQLAuthority.bak'
WITH CREDENTIAL ='AzBackupCred'

And the error message is below

Msg 3271, Level 16, State 1, Line 1
A nonrecoverable I/O error occurred on file “https://sqlprodbkups.blob.core.windows.net/sqldbbackups/SQLAuthority.bak:” Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (403) Forbidden..
Msg 3013, Level 16, State 1, Line 1
BACKUP DATABASE is terminating abnormally.

If you notice the error 3271 can come because of multiple reasons. An important piece of the error message is the error returned by the remote server. In this case, we are getting error 403 – Forbidden. Below is one earlier blog where I discussed Error – 400 – Bad Request SQL SERVER – Backup to URL Fails in Azure Resource Manager (ARM) Deployment Model with Error – (400) Bad Request

WORKAROUND/SOLUTION

As you can see there is nothing wrong with the command. As soon as we run the command, SQL Server uses the credential to connect to storage and performs a backup. When I checked the credential, it was created using Access Keys in the portal. In our case, we have given container name instead of storage account name while creating credential.

Below image should give a clear picture of what needs to be given at which place.

Credential Name = any name which we need to use in BACKUP command

Identity = Storage Account Name which can be found under “Access Keys”.

Password and Confirm Password = Either Key1 or Key2.

It didn’t take much time to realize that my client has used some random name in Identity instead of storage account name and that was the cause of the error. We were able to fix that using ALTER command as below

USE [master]
GO
ALTER CREDENTIAL [AzBackupCred] WITH IDENTITY = N'sqlprodbkups', 
SECRET = N'Value In Key1'
GO

Hope this would help you in fixing error 403, forbidden while taking backup to URL.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Fix Error 3271: A nonrecoverable I/O error occurred on file. The remote server returned an error: (403) Forbidden