Blog Posts by greg_shields

Ask a question

Taming the Three-Headed Beast Kerberos

In a Microsoft Active Directory environment, authentication is obviously critical. Windows domain members use something called Kerberos authentication and if it isn't working properly then likely the domain member isn't either. Most of the time Kerberos just works, but when it doesn't, you need to be prepared put it in its place.

First, it will help troubleshooting if you understand what Kerberos is and how it works. Kerberos takes its name from Greek mythology. You may also know it from its Latin spelling Cerebus. Kerberos is the three headed dog that guards the entrance to the underworld. As a protocol developed at MIT in the 1980's, it similarly employs 3 'heads'.

Figure 1 Cerebus by William Blake

In the Windows world, the three 'heads' are the client, a server and a trusted third party. The latter is a resource that both the client and server trust, which in Windows will be a domain controller. Here's a simplified version of how this works.

Alice wants to talk to Bob at a party, who is rather protective of his privacy. But Alice and Bob both trust Charlie. Alice enters the room, gives Charlie the secret handshake and asks for a pass she can use later to get introductions to talk to Bob. Charlie recognizes Alice and gives her a time sensitive pass, signed by Charlie. After a drink or two of 'courage' Alice decides to talk to Bob. So she goes back to Charlie and shows him the card he provided earlier (Charlie has short term memory problems apparently). She asks for an introduction ticket she can use to talk to Bob. Charlie obliges with another time sensitive document he has signed along with an encrypted secret that only he and Bob know. Alice goes to Bob and offers this ticket. Bob looks at the ticket. Sees that it is still valid, that it has been signed by Charlie and has their secret code, so he knows it came from Charlie. Well, any friend of Charlie is a friend of mine so Bob accepts Alice's offer to talk. If their conversation runs long, Bob might start getting suspicious so he'll politely ask Alice to check in with Charlie again to get another ticket. It is also possible for Alice to ask Bob to prove his identity, and Bob can oblige by returning a secret handshake.

In the Windows world Charlie is the domain controller. Without going into all the nitty gritty details involving public key infrastructure, encryption and protocols, when a user authenticates to the domain controller, she receives a special Ticket Granting Ticket (TGT). When the client wants to communicate with another domain member, she presents the TGT to the domain controller and asks for a service ticket to the server. The service ticket is then passed to the member server which verifies the ticket data, and if all is well, accepts a client server session.

The primary service on the domain controller that manages all of this is the Key Distribution Center or KDC. Figure 2 provides a high level summary of the authentication process.

Figure 2 Kerberos in a Nutshell

As you can see there are a lot of moving parts, which means plenty of places where something could go wrong. Fortunately, if you have a well-designed and maintained Active Directory infrastructure Kerberos-related problems should be rare. But if you suspect an authentication problem, here are some steps you can take.

First off, make sure the KDC service is running on your domain controllers. You can manually check using the Services management console, or use PowerShell. Here's how I can check using the Microsoft Active Directory provider.

PS C:\> import-module ActiveDirectory
PS C:\> Get-ADComputer -filter * -SearchBase "OU=Domain Controllers,DC=globomantics,DC=local" | foreach { get-service KDC -ComputerName $_.Name} | Select Status,Name,Machinename
Status Name''''''''''''''''''''''''' MachineName
------ ----''''''''''''''''''''''''' -----------
Running KDC'''''''''''''''''''''''''''' CHI-DC01
Running KDC'''''''''''''''''''''''''''' CHI-DC02

You should also verify that the SRV records for the Kerberos service are correct from the client. Open a command prompt and start NSLookup in interactive mode. Set the record type to SRV and then query for the record _kerberos._tcp.dc._msdcs.. Here's what I run in my Globomantics domain.

Default Server:' chi-dc01.globomantics.local
> set type=SRV
> _kerberos._tcp.dc._msdcs.globomantics.local
Server:' chi-dc01.globomantics.local
_kerberos._tcp.dc._msdcs.globomantics.local'''' SRV service location:
priority'''''' = 0
weight'''''''' = 100
port'''''''''' = 88
svr hostname'' = chi-dc01.globomantics.local
_kerberos._tcp.dc._msdcs.globomantics.local'''' SRV service location:
priority'''''' = 0
weight'''''''' = 100
port'''''''''' = 88
svr hostname'' = chi-dc02.globomantics.local
chi-dc01.globomantics.local'''' internet address =
chi-dc02.globomantics.local'''' internet address =
> exit

Those are my domain controllers at those IP addresses so there are no problems here.

Because Kerberos relies heavily on time stamping, it is imperative that domain members be configured with an authoritative time source and that everyone is in synch. Kerberos should allow a few minutes leeway but if clocks are skewed more than that, then the tickets issued from the KDC won't be worth the bits they are printed on.

The next tool to become familiar with is NLTEST.EXE. This is a multi-purpose command line tool for querying domain and domain controller configurations. You can use it to find a domain controller that offers the KDC service. NLTEST.EXE should be part of Windows 7. Open a command prompt and type something like this:

C:\>nltest /dsgetdc:globomantics /kdc
DC: \\CHI-DC02
Address: \\
Dom Guid: 44e3c936-5c8f-40cd-af67-f846c184cc8c
Forest Name: GLOBOMANTICS.local
Dc Site Name: Default-First-Site-Name
Our Site Name: Default-First-Site-Name
The command completed successfully

Or you can have it query DNS for the KDC server records.

C:\>nltest /dnsgetdc:globomantics.local /kdc
List of DCs in pseudo-random order taking into account SRV priorities and weight
Non-Site specific:
The command completed successfully

This should give you the same results from the NSLookup example I mentioned earlier. Just remember to use the fully qualified domain name.

The big dog of domain testing is the command line tool DCDIAG.EXE. For our purposes we can use it to verify the proper services are running on a domain controller, which includes the KDC. This utility should also be found on your Windows 7 desktop.

C:\>dcdiag /test:services /s:chi-dc01
Directory Server Diagnosis
Performing initial setup:
* Identified AD Forest.
Done gathering initial info.
Doing initial required tests
Testing server: Default-First-Site-Name\CHI-DC01
Starting test: Connectivity
......................... CHI-DC01 passed test Connectivity
Doing primary tests
Testing server: Default-First-Site-Name\CHI-DC01
Starting test: Services
kdc Service is stopped on [CHI-DC01]
......................... CHI-DC01 failed test Services
Running partition tests on : ForestDnsZones
Running partition tests on : DomainDnsZones
Running partition tests on : Schema
Running partition tests on : Configuration
Running partition tests on : GLOBOMANTICS
Running enterprise tests on : GLOBOMANTICS.local

I used it to query a specific domain controller and I can see that I have a problem with the KDC service which I will have to look into. This is a handy tool you can use to query all domain controllers in the site, but you need to specify at least one.

C:\>dcdiag /test:services /test:dns /s:chi-dc01 /a /v /f:dcdiag-results.txt

With this command I decided to also run some DNS tests while I'm at, get verbose details and save the results to a text file.

The last domain specific task is to double check that the service account for the KDC is still there and is disabled. You can either use the PowerShell module:

PS C:\> get-aduser krbtgt
DistinguishedName : CN=krbtgt,CN=Users,DC=GLOBOMANTICS,DC=local
Enabled'''''''''' : False
GivenName'''''''' :
Name''''''''''''' : krbtgt
ObjectClass'''''' : user
ObjectGUID''''''' : 87a43158-929d-4cbb-b4a9-e4a2bf83a9fb
SamAccountName''' : krbtgt
SID'''''''''''''' : S-1-5-21-2552845031-2197025230-307725880-502
Surname'''''''''' :
UserPrincipalName :

Or the command line tool DSGet.

C:\>dsquery user -name krbtgt | dsget user -L
dn: CN=krbtgt,CN=Users,DC=GLOBOMANTICS,DC=local
desc: Key Distribution Center Service Account
samid: krbtgt
dsget succeeded

It would be odd if anything happened to this account and if it did I would expect you to have problems with the KDC service. Still, it doesn't hurt to be thorough.

On the client side, the best tool you have in your troubleshooting tool kit is another command line tool called KLIST.EXE. The first way to use it is to retrieve information about the user's TGT. The command doesn't require any special privileges, nor do you want it to. You want to see what things look like from the user's perspective.

On a Windows 7 box, Jack Frost is logged on, opens a command prompt and runs KLIST.EXE with the TGT parameter.

C:\Users\jfrost>klist tgt
Current LogonId is 0:0x49f68d
Cached TGT:
ServiceName''''''' : krbtgt
TargetName (SPN)'' : krbtgt
ClientName'''''''' : jfrost
DomainName'''''''' : GLOBOMANTICS.LOCAL
Ticket Flags'''''' : 0x40e00000 -> forwardable renewable initial pre_authent
Session Key''''''' : KeyType 0x12 - AES-256-CTS-HMAC-SHA1-96
: KeyLength 32 - 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
StartTime''''''''' : 12/20/2011 15:29:06 (local)
EndTime''''''''''' : 12/21/2011 1:29:06 (local)
RenewUntil'''''''' : 12/27/2011 15:29:06 (local)
TimeSkew'''''''''' :''' 0:00 minute(s)
EncodedTicket''''' : (size: 1099)
0000' 61 82 04 47 30 82 04 43:a0 03 02 01 05 a1 14 1b' a..G0..C........
0010' 12 47 4c 4f 42 4f 4d 41:4e 54 49 43 53 2e 4c 4f' .GLOBOMANTICS.LO
0020' 43 41 4c a2 27 30 25 a0:03 02 01 02 a1 1e 30 1c' CAL.'0%.......0.
0030' 1b 06 6b 72 62 74 67 74:1b 12 47 4c 4f 42 4f 4d' ..krbtgt..GLOBOM
0040' 41 4e 54 49 43 53 2e 4c:4f 43 41 4c a3 82 03 fb' ANTICS.LOCAL....
0050' 30 82 03 f7 a0 03 02 01:12 a1 03 02 01 03 a2 82' 0...............
0060' 03 e9 04 82 03 e5 00 4d:6d e8 f8 e8 06 80 3b f0' .......Mm.....;.
0070' eb 9d 3a 5e 9b 8c b6 c1:46 b4 64 62 ad 72 28 74' ..:^....F.db.r(t

I've truncated the output because there's not much else to see after the first few lines of the encoded ticket itself. But I do get some useful information about when the TGT was issued, how long it is good for and if there are any time discrepancies. This ticket looks pretty good. On the other hand, if you get output like this:

C:\>klist tgt
Current LogonId is 0:0xef2d17
Error calling API LsaCallAuthenticationPackage (Ticket Granting Ticket substatus
): 1312
klist failed with 0x8009030e/-2146893042: No credentials are available in the security package

Then you have a problem. Or, as is the case from this example, the computer I ran this on does not belong to a domain. I would expect a similar result with a domain member experiencing trust relationship issues with the domain. In any event, any thing I try to do that utilizes Kerberos from this desktop is going to fail.

Assuming the TGT is ok, I can run KLIST.EXE without any parameters and get a list of all the current Kerberos tickets


Figure 3 shows the output.

Figure 3 KLIST Kerberos Tickets

It is also possible to wipe out all the tickets and start from scratch. This should happen if you logoff and back on again, or you can purge the Kerberos ticket cache using KLIST.EXE

C:\>klist purge
Current LogonId is 0:0x36786
Deleting all tickets:
Ticket(s) purged!
Current LogonId is 0:0x36786
Cached Tickets: (0)

As you begin accessing network resources, you'll automatically acquire new tickets.

By now, I hope you are realizing that if you are experiencing a Kerberos related problem the most likely culprits are either related to system time or name resolution. In fact, if you query your members and domain controllers for Kerberos errors in the event log you might find an entry like this:

PS C:\> get-eventlog system -source Kerberos -ComputerName chi-dc01 | select TimeGenerated,EntryType,Message | format-list
TimeGenerated : 11/16/2011 10:55:39 PM
EntryType'''' : Error
Message'''''' : The kerberos client received a KRB_AP_ERR_TKT_NYV error
from the server chi-dc02$. This indicates that the ticket
used against that server is not yet valid (in relationship
to that server time). Contact your system administrator to
make sure the client and server times are in sync, and that
the KDC in realm GLOBOMANTICS.LOCAL is in sync with the KDC
in the client realm.

The bottom line is not to neglect searching event logs for Kerberos related errors.

Kerberos authentication is widely used in Windows networks. Most of the time it works just fine with very little effort on your part. But when it stops working, you'll know because Kerberos' bite can be nasty! When that happens, use these tools and techniques to isolate and resolve the underlying cause and tame the beast.

Be the first to comment

Utilities for Group Policy Reporting


Being in the dark kind of stinks.

Not knowing 'what's going on' in your world is a little scary. Things could be working great ' or they could be falling apart. Without having the ability to produce reports in front of you means you're in the dark, and again, that's no fun.

That's where Group Policy reporting comes in. With Group Policy reporting you can quick discern if everything is 'fine and dandy' or 'Uh oh, red alert' time in your environment.

Let's explore two utilities to help take us out of the darkness and bring come clarity to our Group Policy environment.

GPresult command line tool

Windows XP and Windows 7 ship with the gpresult.exe command. Let's see an example of GPresult on Windows XP in Figure 1.

Figure 1: GPresult running on Windows XP

Here, you can see Windows XP reporting about the 'Applied Group Policy Objects' and also 'The following GPOs were not applied because they were filtered out.'

Applied Group Policy Objects is important, because it shows (from this user's perspective) what they appear to be getting. Conversely, the second category 'The following GPOs were not applied because they were filtered out' shows the equally important GPOs they didn't get. A GPO might not apply for a huge variety of reasons, including being blocked, the side being empty (in this case, the user side of the two policies displayed), security issues or lots of other reasons.

So, in short: knowing what you got and actually what you didn't get is equally important.

Running GPresult on Windows 7 however doesn't work right away. Let's take a look at Figure 2.

Figure 2: Running GPresult on Windows 7

If you simply run GPresult.exe on Windows 7, you are prompted to force feed it some command line arguments. To run the equivalent comment that we saw in Windows XP, you need to run GPresult /R.

You'll get a very similar output from Windows 7 as you did in Windows XP with this command.

Group Policy Results inside the GPMC

The GPresult command line tool we just explored is great ' if you're sitting on the machine you're trying to troubleshoot. However, if you want to remotely see what's going on, a better tool might be the Group Policy Results wizard inside the GPMC as seen in Figure 3.

Figure 3: The Group Policy Results Wizard inside the GPMC

Running the Group Policy Results wizard requires that the target machines' firewall is open enough for the request to come thru. So sometimes running the Group Policy Results wizard against a remote machine might yield what's in Figure 4.

Figure 4: RPC server unavailable means that the target machine's firewall is preventing your request

Once the firewall is taken care of though, it's smooth sailing as seen in Figure 5.

Here, you can see a nice graphical report showing similar information. Additionally any known errors are also reported in the 'Policy Events' tab as also seen in Figure 5.

Group Policy reporting doesn't have to be mysterious. It can be enlightening and help you determine where things are going great, or where your computers and users might need a little attention.

Be the first to comment

Methods for Clearing Space on a Hard Drive

Video Transcript

Hi, this is Jeff Hicks. Today I am going to talk about some ways that you can put your hard drive on a diet.

There may be times, especially on older computers where you are running low on disk space but you can't immediately fix the situation permanently with say, a new hard drive. Instead, you need to come up with ways to give yourself some breathing room.' I am going to show you a couple of PowerShell ways of things that you can do to help with that.

One thing we can do is compress files. Now, when you do this with Cleanup Wizard there is an option to compress old files but that compresses everything and some files just are not very compressible.' Instead, what we want is a way to compress specific files that meet some criteria and certainly maybe just in a particular folder.

I have a tool here. It is a PowerShell script that is actually a WinForms script that I am going to run called'''''''''''''''' compress-FileExtention.PS1. This works pretty simply using WMI. I am going to put in the path that I want to query. I am going to search all of the folders in the work directory. I am going to specify a set of files here. I am going to look for .TXT files, .HTML files, .XML files, word docs and spreadsheets. You can also add other criteria. I am going to set this to only find files or compressed files that are greater than 1 KB in size and search. I am going to do a quick test and do List Only. I click compress, I get a little warning. It is not really going to compress it. List Only will just show me what it would have done.' The query is running and now if I want I could look in Review.' Those files look good. I can take off List Only and if I click Compress again and if I click Yes it will go ahead and compress all those files.' For the sake of my demonstration, I am not going to do that.' That is one way, you are not going to necessarily save a ton of space but every little bit helps.

Another thing you might do is clear out old files, especially things in the Temp folder.' I have a script here; I am going to .\ it to put it back in my session here called Remove-File.ps1.' This creates a function called Remove-File which allows you to specify files or folders based on a cut-off age.' The Default folder is the temp folder, so if I do Remove-File 'whatif. if I had led it to it would remove all those files which have been created older than when this computer started up.' That could be quite useful.' As you can see my Temp folder does not have a lot, I could clear one megabyte but you could also pipe in other files.

Let's say that instead of compressing files I want to delete some files from that work folder 'c:\work' ,$env:temp | Remove-File -recurse -hidden -force -cutoff 5/1/2011 -whatif (which I'll use just for demonstration purposes) If I had run the command without -whatif I would have recovered, in this case, just 22 megabytes but as you can see you could expand that to the entire hard drive or a Users folder. Certainly, the longer and bigger the space the more time it will take so you will have to wait for that to run.

This is some quick peeks, I have got more information in the accompanying article and there is also a link to a ZIP file that has these scripts and a few other goodies so I hope you will take some time to take a look at that.' Thank you very much for your time today.

Be the first to comment

3 Myths About Group Policy Preferences

Video Transcript

Hi, this is Jeremy Moskowitz from GPanswers.com and today I am going to show you some myths and facts about the Group Policy Preferences. Let's just get right into it.

In order to create Group Policy Preferences, you will need a modern machine like a Windows 7 or Server 2008 R2 machine. I am going to create a new shortcut. I am doing this for everyone in the whole domain here, so I will click Edit. I like to use shortcuts as my Preference item of choice for demonstration because it generally just works. We are going to pick a URL object www.GPanswers.com and we will pick the location of the Desktop.' What we will do is pick an icon.' We will pick this little world icon here. Now we have set up a new shortcut, so let's talk about some myths and facts.

Now that we have set up a shortcut, let's go to our Windows 7 machine and we will run gpupdate. Myth number one is that Preferences don't work on Windows 7 and also Windows XP, in fact they do. You can see we have got the Icon right there. Let's go over to our Windows XP machine, and let's see the exact same thing. Now, you will note that the Icon is a little bit different. That is because the machine we are creating the Icon from is a little different than the Windows 7 machine.' So we can see that we have the little world icon there for XP and it is a little bit different on Windows 7, but you get the general idea.

Long story short, myth busted number one is that they do not work on XP.' They do work on XP the trick however, you need to have what is called the proper Client Side Extension installed. It is a free download from Microsoft and once it is installed you are ready to go. That is the first myth busted.

The second myth that I want to talk about, is that Preferences are not Policies. Let's understand what that means, Preferences are not Policies. Now, they use the Group Policy Engine. Obviously, we are in the Group Policy Management Editor to do our work here, but they are not actually true Policy.' What does that mean? It means I can take something and throw it right into the Recycle Bin and move on with my life, so they are not true Policy.' What is true, however, is that if you run gpupdate again they will be reinstated, but they are not Policies they are Preferences. Policy means you cannot work around it, Preference means that users can work around your setting. That is myth number two.

Myth number three, however, is what happens if the machine goes offline? Let's go ahead and throw this into the Recycle Bin. I am going to do something you will not be able to see me do off camera, which is I am going to turn off the network connection between my Windows 7 machine and my Domain Controller. I have disconnected the network cables here so now you can see Windows reacts down there that I have no network connectivity. Let me go ahead and run gpupdate again. The myth is that Preferences somehow magically reapply even if you are offline. That is just not possible.' The Group Policy Preferences really have no way of maintaining that state while offline.

Those are three myths.' I hope that helps you understand Group Policy Preferences a little bit more and that is it.' For more information on Group Policy Preferences, please visit me at GPanswers.com. Thank you very much.

Be the first to comment

Group Policy Preferences Myths & Facts

In order to make use of the Group Policy Preferences, you'll need to understand some specifics. In this quick article, we'll explore the following ideas:

  • Management machine on which to create Group Policy Preferences
  • What is meant by Policy vs. what is meant by Preference

Myth 1

I can't deploy Group Policy Preferences to Windows XP machines

Fact: Preferences works on most target machines types

Group Policy Preferences works awesomely on Windows XP, Windows Server 2003, Windows Server 2008, Windows 7, Windows Server 2008 and Windows Server 2008 R2 machines.

Only Windows XP and Windows Server 2003 machines need a client side update, which is easily performed via WSUS.

Figure 1 ' Group Policy Preferences applied on Windows XP

Myth 2

Group Policy Preferences can be ignored by users

Fact: Group Policy Preferences are delivered, and can be 'worked around'

As we will discuss in Myth 4, a user can work around the Group Policy Preferences settings (generally.) However, what's also true is that the directives are usually re-applied within 90 minutes or so. So, to a user, a preference feels like a policy.

Figure 2 ' A shortcut applied through Preferences can be deleted

Myth 3

Group Policy Preferences re-apply offline

Fact: Group Policy Preferences cannot re-apply when there is no domain controller available

In myth 2, we learned that Group Policy Preferences items are re-applied within 90 minutes or so. But this is only true when the computer can make contact to a Domain Controller and re-establish the Group Policy Preferences' directives.

If the user is offline or the Domain Controller is otherwise unavailable, then the Preference is not reapplied.

Figure 3 ' Group Policy Preferences are not applied without a domain controller

Myth 4

Group Policy 'Policy' is the same as Group Policy 'Preferences.'

Fact: The 'engine' is called (and was always called) Group Policy. But there is a difference between Policy directives and Preferences directives.

Policy directives (most items within the 'Policies' node) are forced upon the target user (or computer) and cannot be worked around. There are some exceptions, but that's the general rule of thumb for Policy.

Preferences directives, on the other hand are set by the Group Policy engine ' but then the user can work around the settings (generally.) Again, there are some exceptions, but that's the general rule of thumb for Preferences.

Myth 5

I can use Windows XP to create Group Policy Preferences directives

Fact: You need a modern management machine to create Group Policy Preferences directives

If you want to take advantage of the Group Policy Preferences, you need to start out on the right foot. That is, you'll need to create your GPOs (which contain Group Policy Preferences) with what I like to call a 'modern management machine.'

It's true, you could go back to previous operating systems to create the Group Policy Preferences. But the ideal situation in which to create Group Policy Preferences is on a modern management machine. That is, Windows 7 or Windows Server 2008 R2.

Note that Windows 7 doesn't have the GPMC built in. You'll need to get it as part of the RSAT, or Remote Server Administration Toolkit ' a free download from Microsoft. Here's an example of when you know you're on the right track: editing a GPO means you get both Policy and Preferences when editing either user or computer sides as seen in Figure 4.

Figure 4 - Policy and Preferences appear when you're using the 'right' management machine to create GPOs

Be the first to comment
Showing 1 - 5 of 88 results

Top Contributors

Talk About SCCM 2012