Just like SQL Server, I do consult with MySQL, PostgreSQL, and MariaDB. One of the most popular questions, I often receive during my consulting engagement is that where can one download the sample database for various databases. While we should always test all of our queries with the real workload as well as the real database. However, whenever we are exploring the new feature, it is a good idea that we work with the sample database and build our scripts which we can eventually apply to the production database.

MySQL Sample Database

Just like SQL Server Sample Databases, MySQL also has sample databases and they are listed here: Example Databases. I am personally a very big fan of MySQL database sakila and whenever I am presenting a consulting session or training on performance Tuning, I use this database to demonstrate the capability of the MySQL.

Here is the list of the sample databases and their installation process.

• Sakila Database (documentation| download)
• World Database (documentation | download)
• Employee Data (documentation | download)

Out of all the three databases, Sakila database is my favorite database. The Sakila database was initially developed by Mike Hillyer, a former member of the MySQL AB documentation team, and is intended to provide a standard schema that can be used for examples in books, tutorials, articles, samples, and so forth.

What I really like about the Sakila database is that it contains the sample views, stored procedures, and triggers. Here is the instructions to install the sakila database on your server as well as the schema of the sakila database.

Let me know if you use any of the sample databases or build your own database. I would be interested to know what is your practice for the databases. Here is the link to how you can download and install the SQL Server sample database WideWorldImorters.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on MySQL – Download Sample Database – Sakila, World, Employee

One of my clients contacted me for an error that appeared while performing a distributed transaction over a linked server. The interesting part was that the linked server was local SQL Server, this is also called a loopback linked server. In this blog, I would share my understanding and the solution of Linked Server Error – Msg 3910 – Transaction context in use by another session.

The situation with my client was that when he runs a query that performs a distributed transaction, it fails. Here is the screenshot of the behavior.

Here is the text of the error message.

Msg 3910, Level 16, State 2, Line 1
Transaction context in use by another session.

In the above screenshot, the IP used for the linked server was the IP address for the local machine.

If you ever try to create a linked server for a local server, you will get an error message.

Text: You cannot create a local SQL Server as a linked server.

WORKAROUND/SOLUTION

Here are a few things we should check if we get this error message.

1. Make sure SELECT @@SERVERNAME is giving the right server name.
2. Make sure SELECT * FROM SYS.SERVERS is showing local server name under server_id = 0

If you must fetch data from the local server, there is no real need for creating a linked server. You can use three-part naming (database.schema.object) instead of four-part naming (server.database.schema.object)

I was able to locate old documentation here which says Loopback linked servers cannot be used in a distributed transaction.

Please comment and let me know if you find some other solution. Here are a few additional blog posts which you may find interesting.

• SQL SERVER – Stored Procedure and Transactions
• SQL SERVER – SSMS: Top Transaction Reports
• SQL SERVER – Find Total Number of Transactions on Interval

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Linked Server Error – Msg 3910 – Transaction Context In Use By Another Session

During the recent consulting engagement Comprehensive Database Performance Health Check with my client, one of the DBA was busy with Always On deployment. He asked me if I can help him with some pointers. Since I was able to fix it and learned something, in this blog I would share my learning about fixing error “Failed to create new NBT interface, status 1450 “in cluster log while creating the “IP Address” resource.

The situation was that DBA was creating Always On availability group and he was creating listeners also at the same time. After watching the cluster manager while he uses the Wizard and presses the “Finish” button, we found that it was an IP Address resource that was going to a failed state. Hence, I asked to create only an availability group first and then add the listener later.

Once the availability group got created, we tried creating Client Access Point in cluster manager and as expected, IP address resource failed to come online. We generated Cluster logs to find the exact error message. SQL SERVER – Steps to Generate Windows Cluster Log? The relevant portion of the log is pasted below. I have removed some details to make it easy to read.

ERR [RES] IP Address <Cluster IP Address>: Unable to create new NBT interface, status 1450
ERR [RES] IP Address <Cluster IP Address>: Failed to create new NBT interface, status 1450. Deleting stale NBT device from cluster database
INFO [RES] <Unknown Resource>: Successfully deleted NBT interface information from database, status 0.
ERR [RHS] Online for resource Cluster IP Address failed.
WARN [RCM] HandleMonitorReply: ONLINERESOURCE for ‘Cluster IP Address’, gen(96) result 5018/0.
INFO [RCM] Res Cluster IP Address: OnlinePending -> ProcessingFailure( StateUnknown )
INFO [RCM] TransitionToState(Cluster IP Address) OnlinePending–>ProcessingFailure.

WORKAROUND/SOLUTION – NBT Interface

Line # 21 and 2 show the actual error, which is what I mentioned in the blog title. Whenever I see the NBT interface in the error message I always remember NetBIOS over TCP (NetBT). Here is the screen is taken from the internet which shows the setting. This is the properties windows on IP address resource in the cluster.

To solve this problem, we disabled “Enable NetBIOS for this address” and after that, we were able to bring IP address online and the issue was resolved.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Always On Listener Not Coming Online – Failed to Create New NBT Interface, Status 1450

As a part of my work, I also provide consultancy to fix the issues. It has three advantages, first one is that I earn money and the second one, more important one, is that I get to troubleshoot and learn, and the third one is I share it further via my blog. In this blog, I would share my learning about the situation where my client was not able to add a database in an availability group and it was failing with error: This database is encrypted by database master key, you need to provide the valid password when adding it to the availability group.

My client said that he can’t add a database to an existing Availability Group. It kept on asking for a password, but the database doesn’t have TDE enabled. He also checked and found that no encryption used. Interestingly, this database as part of an availability group and due to some reason, they have to take it out and trying to add it back. The error message is as follows:

Here is the text of error message:

This database is encrypted by database master key, you need to provide valid password when adding it to the availability group.

Whenever I have such a situation, I always use a profiler to understand what the application is doing to get the details. In the profiler, I could see the below query.

SELECT
c.create_date AS [CreateDate]
FROM
sys.symmetric_keys AS c
WHERE
(c.symmetric_key_id = 101)

When I executed the same query in SSMS, we got a row for this database.

The name of this key was ##MS_DatabaseMasterKey## which means that there was a database master key created in the database, which is causing this issue.

WORKAROUND/SOLUTION – Database Master Key

First, make sure that the database is not having TDE enabled and they are no other encryption used by this database. Here is the command which we used to fix the issue.

Use [DatabaseName]
GO
Drop Master Key

After this, they were able to add the database to AG.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Always On Error: This Database is Encrypted by Database Master Key, You Need to Provide Valid Password When Adding it to the Availability Group

One of my clients, contacted me to fix one of the errors which they received while starting SQL Service. In this blog, I would share my learning about error “Unable to allocate enough memory to start ‘SQL OS Boot’. Reduce non-essential memory load or increase system memory” and how to fix it.

My customer was following the article as had a monster server of close to 2 TB RAM. Tuning options for SQL Server when running in high-performance workloads

They thought that “Large Pages” allocation would help them in a performance boost. Hence, they added trace flag 834 in SQL Server Startup Parameters and tried to restart SQL Service. Unfortunately, SQL Service failed to start, and they contacted me for quick assistance as their business was down.

I asked them to have a look at SQL Server ERRORLOG SQL SERVER – Where is ERRORLOG? Various Ways to Find ERRORLOG Location

Few last lines in the ERRORLOG file were as below:

Using large pages in the memory manager.
Page exclusion bitmap is enabled.
Large page allocation failed during memory manager initialization
Failed to initialize the memory manager
Failed to allocate pages: FAIL_PAGE_ALLOCATION 2
Error: 17138, Severity: 16, State: 1.
Unable to allocate enough memory to start the ‘SQL OS Boot’. Reduce non-essential memory load or increase system memory.

In the event log, it is logged as below.

Log Name:      Application
Source:        MSSQLSERVER
Event ID:      17138
Task Category: Server
Level:         Error
Keywords:      Classic
User:          N/A
Description:
Unable to allocate enough memory to start ‘SQL OS Boot’. Reduce non-essential memory load or increase system memory.

WORKAROUND/SOLUTION

I did some more online study about “large pages” and found that when SQL uses large pages, it would try to grab the complete size of max server memory during startup itself. If it can grab some reasonable amount of memory and can’t allocate further, it would continue to run with a value lower than max server memory. If it can’t grab even the minimum amount of memory, it would raise an error “Unable to allocate enough memory to start ‘SQL OS Boot’”. This can be caused due to memory fragmentation due to the server running for a long time and there are other applications running on it.

To fix the problem, first, make sure max memory is not more than the RAM. It would be a good idea to restart the operating system so that complete unfragmented memory is available for SQL Server to consume.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Unable to Allocate Enough Memory to Start ‘SQL OS Boot’. Reduce Non-essential Memory Load or Increase System Memory

Sometimes I get amazed by the speed at which the search engine crawls and gets the right results. A few days ago, I wrote a blog about a UI error in Always On availability group. Someone found that blog, followed it, got another error and contacted me for assistance. In this blog, I would share my knowledge about how to fix error 15580 – Cannot drop the master key because dialog “GUID” is encrypted by it.

This is the blog that I wrote a few days ago. SQL SERVER – Always On Error: This Database is Encrypted by Database Master Key, You Need to Provide Valid Password When Adding it to the Availability Group

One of my blog readers was getting the same error while using the wizard so he ran below command.

USE [DatabaseName];
GO
DROP MASTER KEY
GO

But we received the error:

Msg 15580, Level 16, State 1, Line 1
Cannot drop master key because dialog ‘0BAF40FB-6CE1-4F73-A6FB-0E57EA4D5B6F’ is encrypted by it

WORKAROUND/SOLUTION – GUID

We went further to figure out where the GUID was coming. Executed below to find if the service broker is enabled.

SELECT is_broker_enabled, name
FROM sys.databases

But found that the service broker was not enabled for the database having a problem. Executed below

SELECT *
FROM sys.conversation_endpoints

and found the conversation endpoint has the entries with conservation_id ‘0BAF40FB-6CE1-4F73-A6FB-0E57EA4D5B6F’. This means there were some leftover entries and it would need a cleanup. This should be done when you are sure that there is no encryption used in the database and server.

USE [DatabaseName]
GO
ALTER MASTER KEY FORCE REGENERATE 
WITH ENCRYPTION BY PASSWORD = 'EnterStrongPasswordAndRememberIt'
GO

After this, we were able to drop master key (same command as earlier)

USE [DatabaseName]
GO
DROP MASTER KEY
GO

And, as mentioned in my other blog, now they were able to add a database to the availability group. Let me know if you have ever faced this issue before and if you have, how did you solved the problem. Please leave your feedback in the comments section.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Error 15580 – Cannot Drop Master Key Because Dialog “GUID” is Encrypted by It

As a part of my work, I also provide consultancy to fix the issues. In my recent engagement with a client, they were getting an error while installing SQL Server 2019. In this blog, I would share the steps to identify and fix error – The account running SQL Server Setup does not have administrator rights on the computer. To continue, use an account with administrator rights.

Let me first say that the error message above was appearing even if the logged-on account was an administrator on the server. In fact, my client also tried with Domain Admin account but still, they were getting error. Since it was a weird error, they contacted me to figure out what is going on.

Here is what we saw on the screen.

I checked SystemConfigurationCheck_Report.htm file which has the exact same error.

The rules which failed were:

1. HasSecurityBackupAndDebugPrivilegesCheck
2. ThreadHasAdminPrivilegeCheck

I have copied and pasted the description below.

And that doesn’t help because they were very sure that the account was a local admin and have all permissions. I asked them to share the setup log files as per Microsoft documentation.

View and Read SQL Server Setup Log Files

I searched for same failures which we saw in html and found below

Slp: Loading rule: ThreadHasAdminPrivilegeCheck
Slp: Creating rule target object: Microsoft.SqlServer.Configuration.SetupExtension.FacetAdminCheck
<>
Slp: Loading rule: HasSecurityBackupAndDebugPrivilegesCheck
Slp: Creating rule target object: Microsoft.SqlServer.Configuration.SetupExtension.FacetPrivilegeCheck

Then searched for FacetAdminCheck and FacetPrivilegeCheck and found these lines.

Slp: Initializing rule      : Setup administrator
Slp: Rule is will be executed : True
Slp: Init rule target object: Microsoft.SqlServer.Configuration.SetupExtension.FacetAdminCheck
Slp: Unable to load DLL ‘sqlsccn.dll’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Slp: at Microsoft.SqlServer.Configuration.SetupExtension.FacetAdminCheck.SccCheckAdminPrivilege()
at Microsoft.SqlServer.Configuration.SetupExtension.FacetAdminCheck.Microsoft.SqlServer.Configuration.RulesEngineExtension.IRuleInitialize.Init(String ruleId)
at Microsoft.SqlServer.Configuration.RulesEngineExtension.RulesEngine.Execute(Boolean stopOnFailure)
Slp: Rule initialization failed – hence the rule result is assigned as Failed
Slp: Send result to channel : RulesEngineNotificationChannel
Slp: Initializing rule : Setup account privileges
Slp: Rule is will be executed : True
Slp: Init rule target object: Microsoft.SqlServer.Configuration.SetupExtension.FacetPrivilegeCheck
Slp: Unable to load DLL ‘sqlsccn.dll’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)
Slp: at Microsoft.SqlServer.Configuration.SetupExtension.FacetPrivilegeCheck.SccCheckPrivilege(String privilege)
at Microsoft.SqlServer.Configuration.SetupExtension.FacetPrivilegeCheck.Microsoft.SqlServer.Configuration.RulesEngineExtension.IRuleInitialize.Init(String ruleId)
at Microsoft.SqlServer.Configuration.RulesEngineExtension.RulesEngine.Execute(Boolean stopOnFailure)
(12) 2020-03-02 09:38:44 Slp: Rule initialization failed – hence the rule result is assigned as Failed

The shows that the real failure was “Unable to load DLL ‘sqlsccn.dll’: The specified module could not be found. (Exception from HRESULT: 0x8007007E)”

Oh my god! I never knew that setup logs can help that much to pinpoint the issue.

WORKAROUND/SOLUTION

When I search for the file sqlsccn.dll, I found that it was available on my local media of SQL Server 2019 but not on the client’s media. In fact, when I compared the total files on my developer media with their media, there were many files missing. In short, this was an issue with missing files from the setup media.

I asked then to re-download the media and with the right media, SQL installation worked like a charm.

Have you ever found such information in setup logs?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Install Error – The Account Running SQL Server Setup Does Not Have Administrator Rights On the Computer. To Continue, Use an Account With Administrator Rights

One of my clients contacted me and asked me the reason for this message? And my first reaction was, did you keep the database in very cold weather? It might be freezing over there and that’s why SQL is saying:   “I/O is frozen on database” and “I/O was resumed on database”? Did you check the air conditioner temperature?

Jokes apart. Let’s discuss the message. Let me paste the complete message and I am showing only one such combination.

This sequence of messages would be present for all databases which got backed-up. I had only four system databases on my test server. I fired disk shadow to take a backup of C drive and once the backup was completed, we can see three such pairs, each with the same four messages. Since we can’t take the backup of tempdb, we are seeing only master, model, and msdb. I have highlighted them with a different color so that it is easy to focus.

THE ANSWER – Frozen

Four messages which are shown in this blog are generated when there is any software taking volume-level backup. I have used DIskshadow to take C drive backup and all the database for which the files were in C drive got backed up. These software use VSS (Volume Shadow Copy Services) which interacts with SQL Server Writer to get backup of those files. The key part is the backup type mentioned in message # 3 – VIRTUAL_DEVICE = {GUID}

If you are seeing such messages, they are information messages and generally can be ignored unless the backups are causing issues with the application. These messages indicate that SQL Server has frozen the IO so that snapshots can be taken for the volume. This is coordination between VSS and SQL Server so that consistent database backup is taken across the volumes. If you are using software like BackupExec, Veeam, Microsoft Data Protection Manager (MSDPM), Azure Backups then it is normal to see these messages.

Are there any such interesting messages which you can share?  Please comment and let me know.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – What do These Messages Mean – “I/O is Frozen on Database” and “I/O was Resumed on Database”?

While trying to try PolyBase on my laptop, I was getting an error. When I looked further, the solution was so different and the error message has no relation to it. In this blog, I would share my findings of error Msg 46530 – External data sources are not supported with type GENERIC.

Here is the query which I was executing:

CREATE EXTERNAL DATA SOURCE BigPinalSQLExternalData
WITH (
LOCATION = 'sqlserver://BigPinal:1433',
PUSHDOWN = ON,
CREDENTIAL = RemoteCredentials
); 

and the error message was

Msg 46530, Level 16, State 11, Line 1
External data sources are not supported with type GENERIC.

Below is the screenshot.

WORKAROUND/SOLUTION – External Data Sources

Based on my internet search, I found that the error message appears when you don’t have a PolyBase feature installed or it is not enabled via sp_configure.

To check if the feature is installed, you can use below T-SQL.

SELECT SERVERPROPERTY ('IsPolyBaseInstalled') AS IsPolyBaseInstalled; 

If you get zero as output, then the feature is not installed at all. The next step is to install the feature using setup.exe. Here is the feature which you need.

If you get the output as one (1) then you need to enable PolyBase using sp_configrure.

exec sp_configure @configname = 'polybase enabled', @configvalue = 1;
RECONFIGURE;

If the feature is not installed and try running above would cause this error

Msg 46923, Level 16, State 1, Procedure sp_configure, Line 166 [Batch Start Line 8]
PolyBase feature is not installed. Please consult Books Online for more information on this feature.

The command would run successfully when the feature is installed.

Did this blog help you in fixing the issue? If yes, please comment and let me know.

Here is the updated list for March 2020 of all the tools which I recommend. Do let me know if you use any of the above tools: SQL SERVER Tools I Use and Recommend (Updated: March 2020).

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – PolyBase Error Msg 46530 – External Data Sources Are Not Supported With Type GENERIC

SQL Always-On feature is so tied to cluster that sometimes you would need to look at various logs to fix a failure. In this blog I would share my learning about fixing error LogonUserExEx fails for user & GetToken – Logging on as the CNO failed with error 1326.

Let me explain the situation and the steps I took to fix the error. The issue was that the listener network name resource was not coming online in the cluster manager. When I checked the properties of the resource, I found the below messages.

On the properties window, I have highlighted these messages.

DNS Status: The handle is invalid.

Kerberos: The user name or password is incorrect.

I asked my client to generate a cluster log. Here is the article which you can follow to generate the logs. SQL SERVER – Steps to Generate Windows Cluster Log?

In the cluster log, the relevant lines are below.

INFO [RES] Network Name: [NNLIB] PopulateKerbKDCLookupCache – DC flags 0
INFO [RES] Network Name: [NNLIB] LsaCallAuthenticationPackage success with a request of size 96, result size 0 (status: 0, subStatus: 0)
INFO [RES] Network Name: [NNLIB] Priming local KDC cache to \\dc.domain.com for domain label ad
INFO [RES] Network Name: [NNLIB] LsaCallAuthenticationPackage success with a request of size 82, result size 0 (status: 0, subStatus: 0)
WARN [RES] Network Name: [NNLIB] LogonUserExEx fails for user SQLAUTHCLU01$: 1326 (useSecondaryPassword: 0)
WARN [RES] Network Name: [NNLIB] LogonUserExEx fails for user SQLAUTHCLU01$: 1326 (useSecondaryPassword: 1)
INFO [RES] Network Name: [NNLIB] Logon failed for user SQLAUTHCLU01$ (Error 1326), DC \\dc.domain.com, domain domain.com
ERR [RES] Network Name: [NN] GetToken – Logging on as the CNO failed with error 1326
ERR [RES] Network Name : Dns: Obtaining token threw exception, error 6
ERR [RES] Network Name : Dns: Failed DNS registration with error 6 for Name: SQLAUTHCLU01 (Type: Singleton)

We can see two errors in above log, 1326 and 6. The meaning of those is matching with what was shown in the UI.

WORKAROUND/SOLUTION – GetToken

I search at various places on the internet and found that this can be fixed by repair of the computer objects. Below is the screenshot for that.

After that, I also asked them to restart the nodes and then the listener came online.

Have you seen such errors in cluster logs?

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – LogonUserExEx Fails for User & GetToken – Logging on as the CNO Failed With Error 1326

Sometimes I see weird issues with SQL Servers, and it helps me in understanding how the software works. In this blog, I would share my learning about the cause of Server Authentication mode disabled. If it is disabled, how would you know what mode is used by SQL Server?

Here is the screenshot of my problem statement. Weird, right?

Logically thinking, if a radio button group is greyed out with nothing checked which means the value is not as expected by SQL Server Management Studio (SSMS). I captured profiler and looked at the script which is executed when we click on the “Security” tab under the “Server > Properties” window. Out of the long script, I have made a shorter version which is used to get the value to populate the server authentication. Below is the secret sauce which populates the radio button.

DECLARE @HkeyLocal NVARCHAR(18)
DECLARE @MSSqlServerRegPath NVARCHAR(31)
DECLARE @InstanceRegPath SYSNAME
DECLARE @SmoLoginMode INT;
SELECT @HkeyLocal = N'HKEY_LOCAL_MACHINE'
SELECT @MSSqlServerRegPath = N'SOFTWARE\Microsoft\MSSQLServer'
SELECT @InstanceRegPath = @MSSqlServerRegPath + N'\MSSQLServer'
EXEC master.dbo.xp_instance_regread @HkeyLocal
	,@InstanceRegPath
	,N'LoginMode'
	,@SmoLoginMode OUTPUT;
SELECT @SmoLoginMode

If the output is 1 then its “Windows Authentication mode” and if the output is 2 then its “SQL Server and Windows Authentication mode”. Feel free to run on your environment and verify it.

WORKAROUND/SOLUTION – Authentication

Now there are two possible reasons, which I can think of, to get the radio button greyed out.

1. If the value returned is something other than one or two OR
2. There is a failure in executing xp_instance_regread, which could be due to permission or any other reason. The best way would be to run the script and see the output.

Now, the next question would be, how to find the authentication mode used by SQL Server, if the option is greyed out? I would always look at the ERRORLOG file to see line having Authentication mode is.

Here are the two possible values which I have seen.

• Authentication mode is WINDOWS-ONLY.
• Authentication mode is MIXED.

If you are running into the same situation, run the script and check SmoLoginMode. If the value is other than 1 or 2 then you have an issue with registry value. Fix the value and then come back to SSMS. If there is an error while running the script, please comment and let me know. You can always reach out to me on twitter here.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Why Server Authentication is Disabled? What Mode is SQL Server Using Currently?

Recently one of my clients contacted me to provide them assistance in finding and fixing the cause of a huge transaction log file. There are many caused to get error 9002 in SQL Server ERRORLOG. In this blog, I would explain the cause and fix of error: The transaction tog for database ‘SQLAuthority’ is full due to ‘REPLICATION’.

I joined a call with them and ran a few queries to know the state of the database. I always start with the below query.

SELECT name, log_reuse_wait_desc
FROM sys.databases
where name = 'Database_Name'

Here is the complete list as of today from books online. Here are the various values which can be seen in log_reuse_wait_desc in sys.databases catalog view.

• NOTHING
• CHECKPOINT
• LOG_BACKUP
• ACTIVE_BACKUP_OR_RESTORE
• ACTIVE_TRANSACTION
• DATABASE_MIRRORING
• REPLICATION
• DATABASE_SNAPSHOT_CREATION
• LOG_SCAN
• AVAILABILITY_REPLICA
• OLDEST_PAGE

In my client’s case, I was seeing “REPLICATION”.

When I queried sys.databases to know about replication using the following query:

SELECT [is_published]
	,[is_subscribed]
	,[is_cdc_enabled]
FROM sys.databases
WHERE name = 'Database_Name' 

I saw this

That was weird! Based on my discussion with them at some point they did try to do a replication. They created a publication but no subscription. When I checked, I found that the log reader job was not running. That explains the cause of the full transaction log.

Fundamentally, SQL Server would keep the data in the transaction log and won’t allow truncation until data is replicated or picked by Log Reader Agent.  You may run into the same issue as there are errors encountered by the log reader agent job.

WORKAROUND/SOLUTION – Full Due to Replication

Since there was no subscriber configured, they were confident enough that we should remove it and it won’t break anything. We attempted to use SQL Server Management Studio to drop the publication, but it was taking a long time. Finally, it got removed but we were still seeing REPLICATION wait. I also used sp_removedbreplication to clean it up. After that, I took transaction log backup and checked log_reuse_wait_desc and it was NOTHING.

After that, I was able to shrink the log file. Here is one of my earlier posts which has a script to shrink the log file of all databases.

DECLARE @ScriptToExecute VARCHAR(MAX);
SET @ScriptToExecute = '';
SELECT
@ScriptToExecute = @ScriptToExecute +
'USE ['+ d.name +']; CHECKPOINT; DBCC SHRINKFILE ('+f.name+');'
FROM sys.master_files f
INNER JOIN sys.databases d ON d.database_id = f.database_id
WHERE f.type = 1 AND d.database_id > 4
-- AND d.name = 'NameofDB'
SELECT @ScriptToExecute ScriptToExecute
EXEC (@ScriptToExecute)

Well, this is how we fixed the Error 9002: The Transaction Log for Database ‘SQLAuthority’ is Full Due to ‘REPLICATION’.

Have you seen such an unexplained situation with SQL Server? Please comment and let me know.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Error 9002: The Transaction Log for Database ‘SQLAuthority’ is Full Due to ‘REPLICATION’

Many of us are working from home nowadays due to Covid-10 (Novel Coronavirus) and during this time, it is very important that we keep watching our health. Many of the readers are of this blog are Database Administrators and Database Developers and it is our job to manage and monitor remote databases. Today in this blog post, I am going to discuss the Work from Home Contest where you can Download Spotlight and Win USD 100.

Spotlight for SQL Server Performance

Every single time when I have to manage my SQL Server remotely, my preference is to monitor it with the help of a spotlight. You should definitely try out Spotlight to:

• Improve SQL Server performance proactively to avoid dropping the ball on your growing responsibilities.
• Gain deep visibility into all aspects of your database environment.
• Diagnose real-time and historical issues quickly.
• Achieve high availability.

When I am busy managing the database in the organization I have lots of different tasks and I have very little time to focus on Performance Tuning and that is where I depend on tools like Spotlight which can tell me immediately with one glance what is wrong with my database and how I can improve the performance of the same almost instantly.

Download and Win USD 100

I recently talked to kind folks at Quest about the DBA’s need to remote manage their database. They have offered 30 days free trial to their flagship product Spotlight. If you download it between today and March 17th, you will be entered to win USD 100 gift card. There are a total of 3 gift cards of each USD 100. So yeah, share the link with your co-workers and encourage them to download Spotlight as well.

Here is the link to Download Spotlight. (Download before April 17)

Three lucky winners will get USD 100 each.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on Work from Home Contest – Download Spotlight And Win USD 100

During the recent consulting engagement Comprehensive Database Performance Health Check with SQL Server 2017/2019, my client’s DBA asked me if I can help with an error message which they are seeing while restoring a backup. This was a very interesting error to me and realize that not many people might have hit this error – Msg 3254 – The volume on device <Path\File> is empty. Followed by Msg 3013 – RESTORE DATABASE is terminating abnormally.

The complete error message which they received was as follows:

Msg 3254, Level 16, State 1, Line 14
The volume on device ‘C:\Program Files\Microsoft SQL Server\MSSQL15.MSSQLSERVER\MSSQL\Backup\ProdDB.bak’ is empty.
Msg 3013, Level 16, State 1, Line 14
RESTORE DATABASE is terminating abnormally.

In my lab environment, I was able to get the same error while I provided a text file which was not a real database backup file.

Command:

RESTORE DATABASE SQLAuthority
FROM DISK = 'C:\Temp\SQLAuthority.bak'

Output:

Msg 3254, Level 16, State 1, Line 12
The volume on device ‘C:\Temp\SQLAuthority.bak’ is empty.
Msg 3013, Level 16, State 1, Line 12
RESTORE DATABASE is terminating abnormally.

WORKAROUND/SOLUTION – Terminating Abnormally

If restore is failing with this error, I would always check the physical size of the backup file first. I have tried various combinations and found that if the file size is less than 512 KB then I was getting the same error. Once file size was touched 512 KB then error message was this:

Msg 3241, Level 16, State 0, Line 15
The media family on device ‘C:\Temp\SQLAuthority.bak’ is incorrectly formed. SQL Server cannot process this media family.
Msg 3013, Level 16, State 1, Line 15
RESTORE DATABASE is terminating abnormally.

The below image shows the timestamp, size and error message change.

From my test, it was clear that the backup file was not a correct file as it can never be less than 512 KB in size. So, if you are getting “device is empty” error, check the size and use another backup of the database which is healthy.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Msg 3254 – The Volume on Device Path\File is Empty. RESTORE DATABASE is Terminating Abnormally

Recently, I wrote an article about COVID-19, Working From Home and Performance Tuning and I got lots of great responses to it. While it is fun to engage with kids and family there are times when we have to manage our jobs. Lots of DBA and Developers who were actively working on new development projects, suddenly realized that their job changed a bit. From developing the new project suddenly most SQL Experts (DBAs and Devs) have to start remote monitoring while working from home.

Monitoring Challenges

Every DBA when they start setting up monitoring for the remote system, the issue which they face is what to monitor in SQL Server. There are so many different aspects of the SQL Server that it becomes pretty impossible to monitor every single aspect all the time. However, if you ask any experienced SQL Server Troubleshooters they will say it is very critical to monitor the following three aspects in any SQL Server.

Workload Analysis / Resource Consumption Monitoring

It is very critical to keep an eye out of your current workload and available resources. The fine balance between both of them will keep your SQL Server healthy otherwise, you will start struggling with a performance of start over providing the resources which is also a waste of fortune.

Incident Playback / Root Cause Analysis

Root cause analysis (RCA) is something I am often hired to investigate. By default SQL Server does not collect all the data to investigate the performance slowness at any point in time. There are few DMVs when we query them they do provide some meaningful data but if due to Server slowness the server had to be restarted, we even lose most of the trace data.

If you want to know what went wrong last time and do proper root cause analysis, you need to have an incident playback feature.

Proactive Monitoring Alert

For any system when things are running fine, no one is often worried. However, when suddenly there are changes in the workload or anything out of normal happens, we need to know immediately. It is critical to know that our system is going to hit the performance bottleneck due to resource unavailability. Proactive monitoring is a good thing but it should be free from false positive.

Call to Action – Remote Monitoring

Here are three calls to action valid till April 17, 2020.

Call to Action 1: Download Spotlight Enterprise and Enter to Win three USD 100

Call to Action 2: Download Spotlight Cloud for Free Performance Monitoring

Call to Action 3: If you have done both action 1 and action 2, just drop the screenshot to pinal @ sqlauthority.com and one of you may win SQL Server Performance Tuning Practical Workshop – Recorded Classes

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Remote Monitoring While Working from Home

One of my clients contacted me for assistance with whom I worked earlier to perform a Comprehensive Database Performance Health Check. They were trying to upgrade SQL Server 2016 to SQL Server 2017 in a clustered environment. First the upgrade passive nodes, it went fine. They failed over to an upgraded node, it also went fine. When they were trying to patch the other node, it failed with an error – There are No More Endpoints Available From The Endpoint Mapper.

I asked them to have a look at SQL Server setup logs and here are the interesting messages.

Detailed results:
Feature: Full-Text and Semantic Extractions for Search
Status: Failed
Reason for failure: An error occurred for a dependency of the feature causing the setup process for the feature to fail.
Next Step: The upgrade process for SQL Server failed. To continue the upgrade process, use the following information to resolve the error. Next, uninstall SQL Server by using this command line: setup /q /action=uninstall /instanceid=SQLQA /features=DQ,FULLTEXT,SQLENGINE,REPLICATION. Then, run SQL Server Setup again.

Component name: SQL Server Database Engine Services Instance Features
Component error code: 0x86D80018
Error description: There was an error to lookup cluster resource types. Error: There was a failure to call cluster code from a provider. Exception message: There are no more endpoints available from the endpoint mapper.

The same message was seen multiple times for other features as well.

WORKAROUND/SOLUTION – Endpoint Mapper

Whenever I face such a situation, I always read the message given by SQL setup very carefully and then follow it. Let’s do it again.

One of the messages is “There was an error to lookup cluster resource types. Error: There was a failure to call cluster code from a provider. Exception message: There are no more endpoints available from the endpoint mapper”. I search for it and found that this could be a generic message when there are multiple versions of SQL Server cluster resource DLLs Next Steps says below:

• The upgrade process for SQL Server failed.
• To continue the upgrade process, use the following information to resolve the error.
• Next, uninstall SQL Server by using this command line: setup /q /action=uninstall /instanceid=SQLQA /features=DQ,FULLTEXT,SQLENGINE,REPLICATION.
• Then, run SQL Server Setup again.

I looked into the registry and found something interesting. There was an entry for SQLQA.INACTIVE. when I searched on the internet about this, there were many bits of advice to run MSIEXEC /X command, clean registry, etc. but I don’t follow them unless I am pretty sure what that would do.

Finally, I went with the command asked by SQL Setup. Executed below from the command prompt. In the current situation, setup.exe referred below is available in the media root from SQL Server 2017.

setup /q /action=uninstall /instanceid=SQLQA /features=DQ,FULLTEXT,SQLENGINE,REPLICATION

Once I ran the command, we rebooted both the nodes and then ran setup again to upgrade SQL Server and it worked like a charm.

Have you ever seen such interesting errors? Please comment and share it with others.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Upgrade Failure – There are No More Endpoints Available From the Endpoint Mapper

One of my clients enabled query store features to perform some troubleshooting by themselves. Since they were not able to find the cause and fix the issue, the contacted me via my famous service Comprehensive Database Performance Health Check. I was able to fix the issue for them and later they wanted to disable query store feature. They contacted me again not to fix perf issues but because they were not able to disable query store features for their database. In this blog, I would share my finding of cause of unable to disable Query Store for Always On Database.

Here is the command which they executed

ALTER DATABASE [PROD_DB]
SET QUERY_STORE = OFF

This query was getting blocked with background threads. These were the values in the command column in sys.dm_exec_requests for those threads.

• QUERY STORE APRC CHECK
• QUERY STORE BACKAGOUND FLUSH DB
• QUERY STORE ASYNC FLUSH

Here was their setting of query store for this database.

I then check “DBCC OPENTRAN” for this database and found one transaction. This SPID was the same SPID which was a background task doing “QUERY STORE BACKAGOUND FLUSH DB”

Transaction information for database ‘PROD_DB’.
Oldest active transaction:
SPID (server process ID): 54s
UID (user ID) : -1
Name : QDS nested transaction
LSN : (1302919:41223:132)
Start time : Mar 26 2020 1:18:49:863PM
SID : 0x0
DBCC execution completed. If DBCC printed error messages, contact your system administrator.

I suspected that this was due to QDS processing its async captures.

WORKAROUND/SOLUTION – Disable Query Store

I searched for Microsoft documentation which might help in query store data flush. I found trace flags about the query store. They are 7745 and 7752.

Here were the steps are done to fix the issue for them.

1. Enabled both trace flags as startup parameters on primary and secondary. No restart yet.
2. Restarted SQL Service on Node2 (secondary)
3. Performed failover of availability group from Node1 to Node2.
4. Executed ALTER command to disable Query Store.
5. Restart SQL Service on Node1.
6. Failed back availability group to Node1 (they wanted AG is run on this node)

By following the above steps, we were able to disable the query store for the database. Have you encountered such a situation? Please comment and share the steps which you have followed.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on SQL SERVER – Unable to Disable Query Store for Always On Database

The reason, I like SQL Server as a product is that there are multiple solutions to one problem and one can use the method they find most convenient. Almost three years ago, I wrote a blog post about how to solve the error: Configuration Manager – Cannot Connect to WMI Provider. You Do Not Have Permission or The Server is Unreachable. The other day, I faced the same error again and this time I solved it with a different method. Let us learn how I did that.

When I was working with my client Comprehensive Database Performance Health Check, I suddenly received the following error:

SQL Server Configuration Manager
—————————
Cannot connect to WMI provider. You do not have permission or the server is unreachable. Note that you can only manage SQL Server 2005 and later servers with SQL Server Configuration Manager.

SOLUTION/WORKAROUND – WMI Provider

Here is what we did to fix the problem. The first opened a command prompt in the location machine and ran the following command according to the SQL Server version which was installed on the machine.

SQL Server 2005

mofcomp “%programfiles(x86)%\Microsoft SQL Server\90\Shared\sqlmgmproviderxpsp2up.mof”

SQL Server 2008 / R2

mofcomp “%programfiles(x86)%\Microsoft SQL Server\100\Shared\sqlmgmproviderxpsp2up.mof”

SQL Server 2012

mofcomp “%programfiles(x86)%\Microsoft SQL Server\110\Shared\sqlmgmproviderxpsp2up.mof”

SQL Server 2014

mofcomp “%programfiles(x86)%\Microsoft SQL Server\120\Shared\sqlmgmproviderxpsp2up.mof”

SQL Server 2016

mofcomp “%programfiles(x86)%\Microsoft SQL Server\130\Shared\sqlmgmproviderxpsp2up.mof”

SQL Server 2017

mofcomp “%programfiles(x86)%\Microsoft SQL Server\140\Shared\sqlmgmproviderxpsp2up.mof”

SQL Server 2019

mofcomp “%programfiles(x86)%\Microsoft SQL Server\150\Shared\sqlmgmproviderxpsp2up.mof”

When you run the above statement, it will give you success output.

However, if due to any reason, you do not get Success, you will have to follow the suggestion provided in my earlier blog post over here.

I hope you will be able to solve your problem easily and quickly and get started with SQL Server Configuration Manager. If you are familiar with any such other tip or trick, do let me know and I will be happy to post about them on this blog post with due credit to you.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

First appeared on SQL SERVER – Fix: Configuration Manager- Cannot Connect to WMI Provider. You Do Not Have Permission or The Server is Unreachable

While working on SQL Server Comprehensive Database Performance Health Check, with a client who uses many different SQL and NoSQL databases. During the consultation engagement, we ended up a very interesting query which actually runs faster on MongoDB and slows on SQL Server. I asked him to show me a Schema for the table in MongoDB, which inspired me to write this blog post on MongoDB Compass – Missing a Schema Section.

MongoDB Compass

If you are using MongoDB, I am very confident that you are familiar that there are two primary tools to interact with MongoDB: 1) MongoDB Compass – Graphical interface and 2) MongoDB Shell – Command Prompt.

As I requested my client to show me a Schema of a MongoDB table, he opened the most popular tool MongoDB Compass and to his surprise, there was a no TAB for Schema. The DBA was really confused and frustrated as every single time he was able to use the Schema Tab which was missing at this point in time.

After a while, he realized that while in a rush, instead of MongoDB Compass, he had opened MongoDB Compass Community, which has limited features. He eventually opened MongoDB Compass and was able to find the Schema tab and we were able to look at the necessary schema. After figuring out the Schema of the table, I was able to help him to improve his SQL Server query, which eventually ran faster on SQL Server. I will write a detail blog post in the future.

Summary – Missing a Schema Section

If you ever find Schema and Validation Tab missing in your MongoDB Compass, it is quite possible that instead of MongoDB Compass, you have opened MongoDB Compass Community which has limited features. You can download MongoDB Compass from this link.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

First appeared on MongoDB Compass – Missing a Schema Section

In 2018, Redgate launched the industry’s first-ever report into the state of SQL Server monitoring. It contained responses from over 600 SQL Server professionals giving a unique insight into how they monitor their estates, the technologies they work with, and the biggest challenges they face. Today we need your opinion.

Last year they released the State of SQL Server Monitoring Report for 2019 which contained around 800 responses. So far, with the help of over 1400 responses from SQL Server professionals, we have learned a lot about where the state of database monitoring is heading.

Last year the focus of State SQL Server monitoring was on the following key topics:

• Organizations are using multiple database systems. While SQL Server was the most used platform, Oracle usage had increased by 20%, and PostgreSQL and MongoDB had also seen a rise.
• Estates continue to grow, with the majority increasing the number of servers they manage and expecting that to continue.
• Monitoring is key to managing large estates, with 60% of organizations using 50 or more servers saying they pay for a monitoring tool.
• Use of cloud technologies is increasing with 15% more companies using cloud-based servers for at least part of their estate.
• Migrations to the cloud and migrations to newer server versions came out as the biggest challenge faced by database professionals over the coming 12 months.

Now, this was the state of SQL Server Monitoring for the last year; but every year is different as the challenges and ecosystem change. This year the aim is to gather insights across database platforms, not just SQL.

2020 Database Monitoring

The year 2020 started with a unique challenge across the world. COVID-19 brought new challenges as many organizations were forced to respond quickly, leading to professionals across the globe working from home. As we adjust to this new form of normal, it has never been more critical to manage and monitor the database remotely. Additionally, as the adoption of the cloud has increased in recent times, the nature of monitoring has also changed significantly.

Call for Action – Opinion

It’s time for our opinion and expertise to be heard again the 2020 State of Database Monitoring survey is now open and will be running until midnight GMT on 30^th April. Upon completing the survey you will be entered into a draw to win a USD 500 Amazon Voucher, you will be amongst the first to receive a copy of the report findings, and Redgate will be donating $5 for each submission towards the World Health Organization’s COVID-19 Solidarity Response Fund.

Reference: Pinal Dave (https://blog.sqlauthority.com)

First appeared on State of Database Monitoring 2020 – Need Your Opinion