Showing posts with label windows server 2003. Show all posts
Showing posts with label windows server 2003. Show all posts

Sunday, January 13, 2013

File Defragmentation Tools for Windows 2003/2008, Redhat/CentOS and Ubuntu

For managing fragmentation of NTFS (Windows Server 2003/2008, XP, Vista, and Windows 7):

For general disk defragmentation, the following utilities offer a substantial improvement in overall performance and efficacy over native operating system tools:
Auslogics Disk Defrag or Raxco PerfectDisk

For use on disks unsupported by the above tools, frequently executed and/or locked files or even a straightforward command line utility that can easily be used as part of a shell script:
Contig from the Sysinternals Suite
Contig has been of particular value when managing backup servers - servers storing huge files with substantial writes on a regular basis. Being able to specify the backup files allows for properly scheduling defragmentation by backup job, and in the process eliminating the need for downtime on these systems as part of this manner of disk maintenance. Can also be used for per-file fragmentation analysis and reporting.

For managing fragmentation of ext4 file systems (newer versions Redhat/CentOS, Ubuntu, Debian, etc):

e4defrag - Linux users (or at least the Linux users I know) have been waiting a long time for the use of an online defragmentation utility. We've all ignored it, pretended as though fragmentation didn't happen on our Linux machines, until the time came for a reboot after 2-3 years of uptime and read/writes forced an fsck that occurred at the worst possible time.

e2freefrag - Provides online fragmentation analysis and reporting by disk, file or mount point.

For managing fragmentation of ext3 file systems  (slightly legacy versions Redhat/CentOS, Ubuntu, Debian, etc)

Good luck! Your options are unfortunately a bit limited.

Many readers may ask: ext3 is a journalled filesystem, why even bother? Primarily, in order to increase IOPS (currently the primary performance bottleneck in terms of price per unit of measurement). Journalled filesystems have seek times just as NTFS does. Reducing those seek times improves performance. Further, unexpected system events can lead to the operating system forcing the journal to be processed. Regular maintenance helps to ensure this process is timely and that downtime is minimized as a result. I have often heard it said that this process "often takes only a second" and as a result can be safely disregarded. While I respect everyone's opinion, I have to very urgently disagree. Most of my experience has been in working in commercial data center environments with several thousand servers. At scale, the statistically insignificant becomes a regular headache. What often happens is part of my concern as an administrator, disaster recovery is just as important in my opinion - safeguarding from improbable catastrophic scenarios and reducing their impact has always been part of my agenda.

That said, let's continue: ext3 requires you to unmount your partition to defragment it. IMO, ext3 is still the most widely used Linux filesystem. I highly recommend the e2fsprogs suite, which includes the following tools:

e2fsck - its just fsck. Not a vulgar typo; performs a filesystem integrity check
mke2fs - creates filesystems
resize2fs - expands and contracts ext2, ext3 and ext4 file systems
tune2fs - modify file system parameters
dumpe2fs - prints superblock and block group info to standard output or pi pe destination of choice
debugfs - a simple memory-only filesystem that can be mounted to perform emergency troubleshooting of your primary filesystem

For defragmentation, you will be typically be using the following:

mount - used to mount and unmount filesystem (also widely known as featuring one of the more chuckle-inducing linux commands when in need of command syntax assistance, #man mount)
fsck - File System ChecK. Checks the specified file system for errors. 
[note: modifying /etc/fstab allows you to specify which devices are mounted]

Some solid non-OS included tools are:


Wednesday, January 2, 2013

Fixing Event ID 10154 - The WinRM service failed to create the following SPN

The Problem

The configuration of the system when this error was encountered is as follows:

A. Windows Server 2008 R2 Redundant Domain Controllers - we will call these and
B. Windows Server 2003 Web Server with Windows Remote Management enabled / part of the Active directory deployment - we will call this
C. For the sake of our example, let's say I have configured an OU named "Web Servers" on those domain controllers

Whenever the Windows 2003 Web server reboots, or WinRM.exe service on the Windows 2003 Web server restarts, the following error was logged into the Event Viewer:

Event ID: 10154
Source: Microsoft-Windows-WinRM
Version: 6.1
Message: The WinRM service failed to create the following SPN: %1.
Additional Data
The error received was 8344: Insufficient access rights to perform the operation.
User Action
The SPN can be created by an administrator using setspn.exe utility.

***NOTE: This issue has also been well documented as occurring while using Windows Small Business Server (SBS) 2003

The Explanation

First its important to understand what all of this means and why we should care. This error and its fix are documented in a number of websites elsewhere, however those documents lack any form of explanation to help us better understand what is occurring here. 

SPN stands for Service Provider Name. SPNs exist on the domain controller to indicate which service applications are assigned to which computers within the Active Directory forest. WSMAN means Web Services Management (notated commonly as WS-Management), which is a Microsoft protocol used to acquire information related to services and applications hosted on a remote server, and to manage those applications and services. WSMAN differs significantly from SNMP by allowing administrators to perform a more comprensive array of tasks. Whereas SNMP would simply get information, WSMAN gets information and allows an admin to remotely install and modify applications based on that information (SNMP has SetRequest, which is limited to a narrow set of predefined variables).

The WinRM service  (Windows Remote Management) is what is installed and runs on servers to listen for WSMAN commands. WinRS (Remote Shell) is the client side application of the protocol, and sends the WSMAN commands to the remote host.

Now that we understand the context of the conflict, we can return to our specific error with a greater understanding of the situation. Its important to note that I was able to verify that the WSMAN SPN does in fact exist on both of my domain controllers, so using setspn.exe to create the SPN wasn't going to help me much. I verified this was true by logging into the domain controllers and running the following command: 

setspn -L WEB 
(remember we are assuming that my webserver is named

The output contained a number of items, including the two I was looking for:

This lets me know that the SPExNs do in fact exist. Knowing that winRM.exe will try to rewrite the SPN every time it starts, and together with the Additional Data field of the error message, we now had a confirmed diagnosis and prognosis - the web server has insufficient permissions to write to the SPN, forced rewriting of the SPN at service start generates the error and while there may be no immediate server-side issues because the SPN already exists, that could change at anytime. 

The Solution

First, it is necessary to confirm that the WinRM service is properly patched and updated. For Windows 2003 servers, the subject of our discussion here, this means updating to version 2.0 provided via KB968930. 2003 does not include WinRM by default, and older 2003 servers that you have inherited may still be running the antediluvian version 1.1. Windows 2008 servers now include version 3. 

Supposing the service is fully updated, there are two ways to go about doing this. Both should accomplish the same thing, but if you have issues with one method try the other. 

The first is the easiest to perform for those more comfortable with a GUI. From your domain controller, launch ADSIEDIT.MSC. Connect to the relevant Active Directory instance (typically just the default local connection is fine), then navigate through the domain to the server we are experiencing this issue with. The order of navigation is:
OU=Variable Organizational Unit
CN=Machine Name
Using our example, I would navigate to:
OU=Web Servers
Right click on CN=WEB and select Properties. Select the Security tab, click Add, "NETWORK SERVICE". (This assumes that you run the WinRM service using the default identity settings - select the account that is relevant for your configuration). Click Advanced and Effective Permissions tab, and select "Validated write to service principal name". Then Click OK to save your changes. Reboot the domain controller and restart the WinRM service.

Once completed, use setspn -L and the Event Viewer to confirm whether the change was successful. If not, you can use the command line option as an alternative: 

dsacls "CN=Web Servers,CN=WEB,DC=ai-host,DC=com" /G "S-1-5-20:WS;Validated write to service principal name"

Same end result here as with the GUI - reboot the DC and restart the WinRM service and check the logs or setspn -L. You're accomplishing the same end result with either task - however there are a host of reasons why a GUI can be problematic. I have yet to encounter a set of circumstances where neither trick does not resolve the issue. If this does not resolve your trouble, please email or comment for me.

Extra Credit

Planning on using the WinRM IIS Extension? Launch Server Manager and select Add Features to provision the needed packages. Reboot your server and launch a command prompt, then use winrm qc to complete the configuration.

Sunday, September 30, 2012

Event ID 1517 / 1524 in Windows Server 2003 Event Viewer - Server Login Requires Reboot

I recently worked on a Windows 2003 server that required manual reboots in order to login, whether via console or Remote Desktop. After rebooting, Event ID 1517 was logged repeatedly in Event Viewer (protip: this error could also appear as Event ID 1524 in Windows Server 2003, or as Event 1000 in Windows Server 2000). Microsoft explains this error in better detail than I can in KB article 944984 - essentially, user profile registry hives are kept in limbo after log off, and never completely terminated. Unfortunately, the hotfix described in the KB article didn't help. I don't administrate this server, I was just called in to fix the issue without breaking anything else. At this point, I know the administrator had an application in need of some coding assistance. I didn't have time to review every application's authentication behavior, though, and I am a novice developer at best.

As an alternative to reviewing miles of code, I installed the User Profile Hive Cleanup Service. The service runs continuously and looks for users that have logged off but still have a registry hive loaded. Each time a user session is terminated, the user, application and effected registry keys are logged in Event Viewer.

There are no fancy configurations needed to install the User Profile Hive Cleanup Service - just download, install and reboot. The service immediately resolved the issue for me, and no further logon issues were encountered. After the first reboot the eventvwr had a helpful entry in the System log showing me which application and user was causing the issue.

Tuesday, April 17, 2012

Websockets and IIS7

So its been about 5 months since the IETF released the RFC 6455 proposal for websockets:

The websocket API is a protocol that allows for the bidirectional transfer of http/https data. This breaks down to a single initial handshake and then autonomous communication from both the server and client concurrently. With it comes a significant performance improvement (as only one handshake is needed, and client-side implementation gets much simpler) and a number of practical applications - I always think chat clients, but the applications are endless for web driven applications that require real time data transfer (HTML5 games that don't suck!)

Its no secret that websockets will not work with a standard IIS7 implementation. Http.sys is a greedy bugger, and gobbles up all connections listening on port 80. Even with WCF, there is no formally recognized workaround besides "wait for 8" and the native websocket/SOAP functionality that it will bring with it.

That being said, some folks may find a need to use websockets technology on an IIS7 server, and I found a neat prototype from Html5 Labs that can do just that:

The protoype is WCF / Silverlight driven.

Haven't had a chance to test it yet, but I have a feeling this will come in handy in the future. Let me know what you think!

[EDIT: HTML5Labs deactivated the download to their prototype last month, sorry guys. I am not sure why - its not like 2008 is going anywhere any time soon. The good news is I am compiling some guides on using WebSockets in Windows Server 2012 here. For those developers who are not in a position to upgrade, it may be worth checking out SignalR as one alternative to websockets. SignalR relies on APM as opposed to WebSockets, and can be of benefit when establishing fast connections - it can be deployed on Server 2008 and using older browsers ]

Tuesday, March 20, 2012

Reinstalling MDAC

Microsoft Access Data Components are usually fairly stable. They tend to be updated with significant OS related updates (I'm looking at you, Service Packs). 

That being said, issues do happen. Today I encountered an issue following a P to V migration using Hyper-V for a Windows 2000 server with an ADO connection to a MSSQL database. Somehow MDAC versions become mismatched during this process.

Your actual error may vary. Your application may throw an error 429 "Active-X component can't create object", you might get a IPP_E_MDAC_VERSION error. In the case today, a line in the website's general.asa was mentioned as an invalid object.

Download and execute the Microsoft Component Checker to verify a mismatch against your required Component versions - Use this for 2003 and this for 2000.

For Windows Server 2003 and 2008 systems, I would typically advise an in-place upgrade as outlined here. You can try a manual install on these OS' but frankly it is not as reliable as an automated repair (or an SP update, if one is available). Once I have enough time to creatively break Windows Server 8 I will hopefully update accordingly.

For Windows 2000, your task is of course a bit more painful. Download the appropriate installer from Microsoft (this is for MDAC 2.8 SP1 - I've tested up to 2.8 using Server 2000 and SQL 2000 and it works, usually the issue is one of your files are older than anticipated by MDAC, going newer hasnt hurt me yet, and is the latest confirmed as working by Microsoft). A formal compatibility list is also available.

After installing and rebooting with Mdac_typ.exe, reboot and your components should now once again be compatible. The most frequent issue I have seen is as the result of running the utility in a non-Administrator account, which leaves out key registry components. Regmon is a bit outside of our scope for this late-night post, but we will get to registry permissions hacking in later posts.

RAT Bastard

Earlier this week, several servers I maintain were targeted by automated attempts to upload a remote access trojan (RAT). The RAT is a simpl...