I Already Googled It

Perhaps you’ve seen this dialog when using Microsoft Outlook:

You may have noticed that occasionally the “[a]lways ask before opening this type of file” is greyed out—as it is in the screenshot above. There’s actually some very interesting Windows heritage to this.

For backwards compatibility (the hallmark of all things weird and interesting in MS Windows) there’s a redundant registry hive in Windows—HKEY_CLASSES_ROOT, otherwise known as HKCR—which presents a “merged” view of HKLM\Software\Classes and HKCU\Software\Classes.  The reason it exists “[f]or backwards compatibility” is that back in the day, when Windows was 16-bit, it wasn’t multi-user—there was no HKCU, since the distinction of multiple users didn’t exist, so it didn’t make sense to have a “current” one, and accordingly HHLM\Software\Classes and HKCU\Software\Classes were one hive—HKCR.

Under HKCR is Windows’ file association data.  There’s a key under it for each file extension—all the keys which begin with a dot—and a plethora of keys with descriptive names.

The information about file types is split out into two keys—the key which represents its file extension, and the key which represents its actual logical type.

The key which represents a file extension has as its default value a REG_SZ which points to the actual logical type of that file extension, and accordingly another key under HKCR.

For example, if you have Chrome installed and setup as your default browser, you should be able to obtain the following from the command prompt:

C:\Users\rleahy>REG QUERY HKCR\.htm /ve

HKEY_CLASSES_ROOT\.htm
    (Default)    REG_SZ    ChromeHTML

C:\Users\rleahy>REG QUERY HKCR\.html /ve

HKEY_CLASSES_ROOT\.html
    (Default)    REG_SZ    ChromeHTML

.htm and .html are two separate file extensions—there are two separate keys for them under HKCR—but both of them have a pointer to the same logical file type—”ChromeHTML“.

UAC means that even if you’re logged in as an administrator, your programs are still running with user-only privileges.  This means that even though all this file association data appears to be in the same hive, it’s not.  Accordingly, Outlook may have the rights to modify some—if they’re defined in HKCU\Software\Classes—and may not have rights to modify others—if they’re defined in HKLM\Software\Classes. This is why the checkbox is always greyed out, and why tutorials elsewhere on the internet tell you to run Outlook as administrator to put these dialogs to rest.

But, editing the registry is a much simpler, much more straightforward, and much more scalable way to change this behaviour.

Under the key which represents the actual logical type of a file—in our example ChromeHTML—there either exists, or may exist, a value called “EditFlags“.  It should be of the type REG_BINARY.  Setting it to 00 00 01 00 will suppress the behaviour, whereas setting it to 00 00 00 00 will reenable the dialogs.

For our ChromeHTML example:

C:\Windows\system32>REG ADD HKCR\ChromeHTML /v EditFlags /f /t REG_BINARY /d 000
00100
The operation completed successfully.

Programs may return a code when they exit.  In C this is the value that main returns.  By convention, success is 0, failure is non-zero.

Apparently, however, the developers of Robocopy — which is otherwise an exemplary and indispensable utility – didn’t get this memo, which caused a problem that plagued me for 2.5 years.

A bit of background here:

When a database is modified, the database engine writes the changes into the database proper, and also logs the change in aptly-named log files.  It’s usual to put the log files on a separate physical volume from the database proper.

This way, if the volume containing the database proper fails, you can take your last backup, merge all the changes recorded in the log files into it, and it’s like nothing ever happened.

The issue here is that the database engine has to know that a backup occurred, at which point it can purge all log files (since you have the database in a consistent state with the transactions in the logs merged in, which makes them irrelevant and unnecessary).

Under the hood a bit here.  Windows has the abilities to make shadow copies, which are basically a snapshot of a drive.  So you take a snapshot — i.e. make a shadow copy — of a drive, and then you can copy off of it while the drive is still being used.

This is a great way to backup production servers.  But what if you take a snapshot with a file to be backed up in an inconsistent state?  And if you create a shadow copy and then copy it, how does a database engine know you created a backup?

For this reason, database engines have a VSS writer, which the shadow copy service can contact and notify about copies.  When you create a shadow copy, the write is notified, and can delay the snapshot until it’s placed the files it controls in a consistent state.  When you use a shadow copy to make a backup, the writers are notified and can purge logs.

The way you can do this simply and easily is with a command line utility present in Server 2008 and Server 2008 R2 — DiskShadow.

DiskShadow can operate unattended by running from a DSH file, which is essentially a script that runs within DiskShadow.

Here’s what one looks like:

set verbose on
#delete shadows all
set context persistent
writer verify {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
set metadata C:\Backup_Scripts\shadowmetadata.cab
begin backup
add volume C: alias SH1

create

expose %SH1% P:
exec C:\Backup_Scripts\exchangeserverbackupscript1.cmd
end backup
delete shadows exposed P:
exit

(A detailed explanation of the above can be found here.)

The important lines are “begin backup” and “end backup”.  These cause VSS writers to be notified.  And assuming success, on “end backup” will cause log files to be purged automatically.

However, when Robocopy returned 1 — causing the script containing it to return 1 also — DiskShadow thought the backup had failed, and therefore my Exchange — which is what was being backed up (its write has the identifier seen above) — thought the backup had failed, and therefore believed that I didn’t have a copy of the database in a consistent state, and therefore kept every log file for 2.5 years, which accumulated to 45 gigabytes.

For comparison, the database itself is only ~5 gigabytes.

The solution?  Filter Robocopy’s return codes, as follows:

robocopy "P:\Program Files\Microsoft\Exchange Server\Mailbox\First Storage Group" "\\leahyfs\J$\E-Mail Backups\Day 1" /MIR /R:0 /W:0 /COPY:DT /B
IF ERRORLEVEL 1 exit /B 0
exit /B 1

You’ve been warned…

Something useful that I use pretty regularly (I have a bootable, labeled thumbdrive for installing both OSes mentioned in the title on my desk at work).  Very useful if you have a laptop — or other computer — with no optical drive — something that’s becoming more and more common.

Probably works with Vista/2008 (non-R2) as well, but I haven’t tried it so I make no claims.

You’ll need a copy of the OS-in-question — mounted ISO or actual disk — and a thumbdrive at least 4 gigabytes in size.

Open the command prompt (as Administrator if you have UAC enabled) and run “diskpart“.

Type “list disk” to get a listing of the drives attached to your system.  Pick out the number of the thumbdrive — you can identify it by its size.

Type “select disk #” where “#” is the number you discovered in the preceding step.

Type “clean”.

Type “create partition primary“.

Type “select partition 1“.

Type “format fs=ntfs quick“.

Type “active“.

Close the command prompt and copy all the files from the disk — or mounted ISO — to the thumbdrive and you’re golden (robocopy is useful for this, use “robocopy X:\ Y:\ /S” where “X” is the disk/mounted ISO and “Y” is the thumbdrive) (you may be able to use WinZip or something equivalent to extract the ISO to your thumbdrive, but I’ve never tried this).

When booting you can either go into the BIOS and set the thumbdrive to first in the boot order, or hit the hotkey (typically F9 or ESC) to get the boot menu and boot off of it once (the latter is the method I prefer).

Worth noting that while some BIOSes claim to support booting off your thumbdrive, they won’t for no reason.  I’ve only had this problem with a netbook, but there may be other culprits out there…

I was attempting to write a script at work today that would alter the default user’s registry, so that each new user created on the machine (by logging in for the first time) would have a RunOnce that would fire the first time they logged in.

The first challenge was finding out where the registry hive that each new user’s HKCU is based off is stored.  That was accomplished easily enough: It’s in C:\Users\Default\NTUser.dat.  You can mount unloaded registry hives using reg.exe load and unload them (after you’re finished modifying them) with reg.exe unload.

The next challenge was that I wanted to copy a file to this new user’s %appdata% directory, but using xcopy just wasn’t working.

The not-so-obvious problem was that when RunOnce runs something, it isn’t running within a proper COM interpreter shell, so things you’d expect to work — like expansion of “%appdata%” into something more meaningful — don’t.

The solution is to give the command a COM shell by invoking it with cmd.exe /c <command>.

Another useful tip: If you want a command window to persist after the command finishes — for debugging, for example — use cmd.exe /k <command>.

So after letting it stagnate/grow haphazardly, I finally decided that it was time to upgrade/reorganize the core of my — perhaps excessively – complex home network.

Here’s some “before” pictures (the rack that I have everything in was stuck in a closet barely wide enough to accomodate it):

As you can see, very crowded, very disorganized. The contortions required to get behind the rack (and thereby do any real work) was ridiculous.

Plus the switching infrastructure was just daisy chained D-Link consumer switches I bought off the shelf.  Not a lot of throughput, a lot of bottlenecks, et cetera.

It was time for a change.

The closet was unfinished, to begin with, so in order to facilatite work in there, the rack had to come out.  However, all the network drops for the whole house came to that closet, so we’d have to run network lines from the closet to the rack’s new location, so I bought 31m of network cable from Home Depot, and colour-coded keystone jacks etc. from Monoprice.

Next was the issue that without the rack there, there was nowhere to put the crappy D-Link switches.  I solved this by buying 2ft patch cords off Monoprice (to patch in the network drops neatly), and by having plywood put up on the wall the patch panel is on now (allowing greater flexibility in surface mounting).

To rectify the switch infrastructure situation, I bought a D-Link DGS-1100-24 (about $300 from NCIX) and a D-Link DGS-1100-16 (about $200 from NCIX).  This blow was softened because I had a $100 NCIX gift card from a friend in Nova Scotia, payment for a Linode (fremont.rleahy.ca/ns1.rleahy.ca) that we share.  These are managed switches, with rackmount ears, that support 802.1Q and LAG (whether 802.1ax or their own proprietary solution — or both – I don’t know).

Surface mounting all of these things in the closet on the plywood allowed me to greatly neaten that area up, while reducing the number of switches by using the VLAN features — one VLAN for LAN, DMZ, and WAN — of the DGS-1100-24.  I ran four lines through the wall to the location of the new servers, which were aggregated and plugged into the DGS-1100-16, which I rackmounted in the rack, giving me 4gbps of bandwidth between the “core” and the servers.  All three VLANs are trunked across this aggregated link.

I took the opportunity to recable the entire rack — using pre-made cable bought cheaply from Monoprice — reducing the spaghetti to something manageable.

Here’s the “after” shots:

Very pleased with the way that it turned out. Looks a lot better, throughput is going to be a lot better, design makes a lot more sense, going to be a lot easier to work on.

Now the trick is keeping it as neat as it is…that’s always the problem…

The best part of the whole thing is that thanks to careful movement of the rack (it had to go down a step, we used a plywood ramp) and the UPS, I didn’t compromise my uptime moving the servers, even though they were disconnected from AC power and the internet:

Thanks to my Dad for helping me move this stuff around. Also thanks to this blog post for letting me know how to get images side-by-side in WordPress.

So I had this problem this week, and after finally doing a combination of finding the problem mentioned in a very obscure place, and diagnosing it myself, I thought I’d post about it in the hopes that someone, somewhere, sometime finds this in a search and isn’t pained in the same way I was.

KB968078 for ISA Server 2006 is evil!  Do not download it, much less install it.  If you’ve installed it, reformat and reinstall.

When installed, KB968078 has the following effects:

• PPTP VPN (both client and site-to-site) stops working.  Forever.  This cannot be fixed.  You’ll get something like this in the event log:

  Event Type: Error
  Event Source: RemoteAccess
  Event Category: None
  Event ID: 20050
  Date: 21/02/2012
  Time: 12:03:44 PM
  User: N/A
  Computer: LEAHYISA1
  Description:The user LEAHYISA1\BaragerHome connected to port VPN3-106 has been disconnected because no network protocols were successfully negotiated.

  For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

• RDP stops working.  Forever.  This cannot reasonably be fixed, although there are workarounds.

I had this happen to me on two installations of ISA Server 2006.  The second time it happened was when I realized it was the hotfix.  Simply uninstalling the hotfix had no effect.

Beware.

C++ is the one area of programming that I’ve lamentably felt lacking, especially the more and more I get involved in programming specifically for Windows.  Whereas Linux/POSIX programming feels designed unequivocally for C, Windows seems to gravitate more towards C++.

So, I took the opportunity—four days off—over the Christmas holidays to learn C++ and C++/CLI.

I also took the opportunity to do something that I’ve been wanting to do for a while—make a fiber library for .NET.

I’m not going to say that it’s complete (still some stuff I want to add to it), but here goes:

Download here

Source and XML file (containing documentation) within.  If you want to compile it use cl.exe with the /clr flag on fiber_all.cpp (which #includes all the other cpp files and the header).  Remember to specify /link /DLL /out:fiber.dll if you’re just compiling the source into a DLL to use with C# or something.

Example command line invocation:

cl.exe /Ox /clr fiber_all.cpp /link /DLL /out:fiber.dll

The /Ox is thrown in to enable optimization.

As an aside: If you’re trying to compile C# code against this with csc.exe, unless you used the 64-bit cl.exe, you’re going to have to go ahead and specify /platform:x86, otherwise your C# app will throw terrible exceptions and die as soon as it starts.

How it works:

All the classes that the library uses I took the liberty of putting in the namespace System.Threading, just for intuitiveness.

.NET threads are silly in their implementation—they don’t map onto native OS threads, and may be implemented as native threads, fibers, some combination of the two, or something else entirely, which means that if you just start up a .NET application, or create a new thread in one, a call to ConvertThreadToFiber may be illegal or dangerous—all-in-all, something you don’t want to do.

The library implements a property—Fiber.IsFiber—which can let you know whenever your code is running within an existing fiber.  If you’re not in an existing fiber, and you create a new fiber, the library creates a new, native thread using CreateThread which it then converts to a fiber and immediately calls the function you provided to the fiber constructor.

Once you’re within a fiber, calling the constructor—without providing the new_thread—will create new fibers not attached to any thread, which can then be safely scheduled using SwitchTo.

The most interesting part is waiting for a group of fibers to finish. If the currently-running fiber “falls through” (i.e. ends and doesn’t call SwitchTo then the underlying thread ends. Since fibers can switch between threads, only the currently running fiber has a thread associated with it. By finding this fiber (the fiber with its Scheduled property set to true, you can get a FiberThreadWaitHandle from the WaitHandle property which will allow you to wait (just a wrapped call to WaitForSingleObject) for the underlying thread to terminate (calls to unscheduled threads will just return null).

Note that the Scheduled property is a bad way of finding the fiber to get the wait handle from, since the fiber could unschedule itself and schedule a different fiber in the time between when you call Scheduled and when you call WaitHandle, leading to a previously-scheduled fiber spitting null back at you.  You should implement your own synchronization (I like to use a ManualResetEvent at the top of my initial fiber function, which waits ’till the main thread releases it after grabbing the wait handle).

When a fiber “falls through” its Finished property will become true, which means that you shouldn’t switch to it anymore, and should call Dispose on it to make sure the unmanaged underlying data types and operating system handles get cleaned up.

Comments, concerns, questions, just e-mail me.

I just realize that when I posted yesterday I overlooked an instance of one flaw of Java exacerbating another flaw.

Yesterday’s post was about Java’s absence of pass-by-reference, and one Java programmer’s (i.e. “hostage’s“) defense thereof.

Talking about the idea of making a class that encapsulates the return values and returning that instead of having multiple return values via pass-by-reference I said:

Moreover, creating objects is non-trivial, especially creating an object so you can copy reference types into it, push the pointer to it onto the stack, pop it off, and then copy the reference types back out.

On further thought, I realized that this problem would be greatly lessened if Java supported something else that it doesn’t—structs—which are far cheaper to create than objects.

I hate Java.

It probably has a lot to do with being a libertarian, since Java is the programmer equivalent of big government—you want to write code, do it our way or not at all.

There are tonnes of dogmatic language decisions made about Java that aren’t made by its competitors—which are syntactically almost identical but much more flexible and useful because of the lack of dogma, e.g. C#—that I could rant about, but the most egregious—the one that I always fall back on—is pass-by-reference.

I was unable to sleep for a bit last night, and bored of watching Criminal Minds (the only non-HBO TV show I watch), so I decided to randomly Bing and see if Java had an equivalent to C#’s ref or out keywords yet.

The answer appears to still be “no“.

It seems, however, that when people use a deficient tool, they become emotionally attached to it and rather than recognizing and admitting its flaws they gloss over or even actively excuse them—a type of programming Stockholm Syndrome it would seem.

Case-in-point.

Let’s go through his workarounds one-by-one.

On a side note, I think it’s interesting that the header for the section I linked to is “[w]ays to avoid needing pass-by-reference“, which suggests that pass-by-reference is undesirable and should be “avoided“, whereas the anchor (if you view source) is “workaround“, which more accurately reflects the state-of-affairs: Something should be there, isn’t, and now you need to hack around it.  A bit of cognitive dissonance/doublethink/denial going on there.

I digress.

If any of your return values are status codes that indicate success or failure of the method, eliminate them immediately. Replace them with exception handling that throws an exception if the method does not complete successfully. The exception is a more standard way of handling error conditions, can be more expressive, and eliminates one of your return values.

Except exception throwing and catching is expensive (it triggers a stack trace, among other things), and it isn’t “a more standard way of handling error conditions“, it’s supposed to handle exceptional conditions (hence the name?). Not all errors are exceptional. If I want to read from a stream, but the stream is at the end of the underlying data, that’s an error, but it’s not exceptional since you reach the end of pretty much every stream (i.e. reaching the end is the rule, not the exception).

Find related groups of return values, and encapsulate them into objects that contain each piece of information as fields. The classes for these objects can be expanded to encapsulate their behavior later, to further improve the design of the code. Each set of related return values that you encapsulate into an object removes return values from the method by increasing the level of abstraction of the method’s interface. For instance, instead of passing co-ordinates X and Y by reference to allow them to be returned, create a mutable Point class, pass an object reference by value, and update the object’s values within the method.

Well sure, except that introducing non-standard classes just because one method had to return two values is idiotic and actually makes the code more confusing and harder to learn (since you have to learn the interface for a new class in addition to the signature/purpose of a new function).

The Point example is obviously hand-picked to make this look like a trivial argument, since the Point class (or struct, depending) is pretty much a given. But what about the case of highly efficient methods like TryParse, which is better than the corresponding Convert method solely because it doesn’t throw exceptions. Should we make a weird Boolean and Int32 encapsulating class just so this method can return without violating sacred Java dogma?

Moreover, creating objects is non-trivial, especially creating an object so you can copy reference types into it, push the pointer to it onto the stack, pop it off, and then copy the reference types back out.

Dumb.

If you find yourself passing in an object and then returning a new version of that object, and the object is mutable, then consider moving the method to be a member of the class of the object that you were passing. This can improve the encapsulation of the design and simplify the interface.

Well sure unless you happen to want both versions.

If, after all steps above, you are still left with multiple returns to a method, split the method into several different methods that each return a part of the answer. Using this set of methods from client code will be much clearer than one mega-method.

Except some functions that return many values are logically one function. Separating them to get around arbitrary dogmatic language restrictions doesn’t make any sense whatsoever and can actually make things less clear because one logical operation now requires more than one function invocation.

Besides, function invocations (unless inlined) represent overhead.

Here’s a quick disclaimer before I begin: I’m not a web designer.  I don’t want to be a web designer.  This picture pretty much sums up why:

On top of the excellent points raised by the above graphic (I actually do have my own keyboard at work.  I don’t “bring” it though, I bought an extra one so I could leave it there and so it’d always be there waiting for me…) there’s also the fact that I really don’t care how things look.  When you think about it, it doesn’t matter either.  Why does it matter if your website has a billion little JavaScript tricks et cetera as long as it gets the point across?

It doesn’t.

In fact, I’ve noticed that how pretty a site is, and the quality of its content, seem to be inversely correlated.

And then there’s the fact that 9 out of 10 “pretty” sites just don’t work.  At all.

Anyway, this post is about the dirge that is “negative space“.  You go to a site, and you see somewhere in the area of seven miles of totally wasted space on either side of the content.  Either these people never understood that you can size things in CSS proportionally (i.e. set a div to 95% of the screen’s width rather than 800 pixels to appeal to the non-existent viewer who’s still trapped on a circa 1997 monitor), or they think that it’s somehow artistic and appealing.

Sure, negative space is an accepted idea in art and photography, but this is web design.  You’re not trying to appeal to the user with art, you’re attempting to present content, and content has to be read or viewed, and to do that screen real estate is required.  What are you hoping to achieve by wasting 2/3rds of the screen?  Is it your mission in life to make your users use the scroll wheel more?

Here’s an example—the Battle.NET forums (inb4: WoW lol) (I’ve just cut out the left half the screen so the image isn’t too wide/large):

(Haha idiot can’t spell “looking“…)

Seriously, what in the name of God? This might be alright if it was just the topic listing that was like this, but the thread display has this same design, so you wind up scrolling something like 3-4 times the normal amount. Do they think that users find longer pages more appealing or impressive?

If the point of your website is to allow users to enjoy your content, let them enjoy your content! Your “content“—hopefully—isn’t so small that you need “tricks” like this to make appear bigger, or merely the design itself, because if either is the case, you’re pretty fucking vapid, and you need to worry about other things than how nice your page looks.

Or maybe you’re worried about users on widescreen monitors having to constantly look side-to-side?

NEWSFLASH: I bought a widescreen monitor for a reason; I actually wanted it to be wide.  Don’t take that choice away from me you asshole, if I wanted to be compulsive scroll wheel user I would’ve bought a 4:3 monitor, but I didn’t so I don’t.

Also: I’m pretty sure all window managers from the Explorer shell to X Server allow you to resize windows.  If I’m so stupid that I don’t realize my eyeballs move side-to-side in their sockets, and my neck gets sore or something else, I’m either too fucking stupid to matter, or I’ll just make the window thinner and put it in the middle of my widescreen monitor and it will look the same!