I guess I should start with an apology.  Faithful readers have probably left my blog in their droves because it’s been so long since I posted anything.

Here’s the thing.

When I worked at Microsoft, my job required that I was always looking at new technology.  It seemed reasonable to me that I should write about it, because other people picking up that technology might gain something from my writing.

These days, I don’t do much with new technology from Microsoft.  My clients tend to be fairly conservative in their technology choices.  While I continue to learn a lot about Microsoft’s platform, I just don’t feel inclined to write about my findings, because Google tells me that a lot of people have been there before me.

Now, I’m the sort of person that always wants to be learning about new stuff.  When I left Microsoft I suddenly felt that I wanted to learn about the wide world of software outside of their platform.  I’ve been learning to develop apps for the Mac, for example.  I’ve been learning to build web apps using PHP, and to run them on Apache on Linux.  I’ve been using MySQL, and I’ve even written a little Java.

I haven’t turned my back on the Microsoft platform.  I can’t see that happening any time soon.  I spend most of my working life using their platform and tools, and I’m quite happy doing so.

Soon I’ll be starting a new blog.  On it, my future posts are just as likely to be about those non-Microsoft technologies as they are to be about Microsoft ones.  In fact, maybe more so.  Because I’m such a novice in those other fields, I find something interesting at every turn.

When my new blog is up-and-running, I will of course post a link here.

In my last post, I talked about how a process' access token determines which user session it belongs to.  With that information, and if I have sufficient privilege, I can programmatically start a process in any running session.

In my scenario I have a highly-privileged service that wants to communicate with a lowly-privileged app running in each established session.  The machine I have in mind is shared, and there might be multiple sessions that are all logged-in.  How do I get the low-privilege process to run?  I could make it a startup app for all users, but I can't really trust that people won't change their startup apps, or that they might not simply kill the process.  I would prefer that my service is able to create the other process and just have it run inside whichever user session it chooses.

Here's some code that does what I need...

...and here's some explanation...

• Use OpenProcessToken to get the access token for the running process (this in my highly-privileged service).
• Duplicate that token with DuplicateTokenEx, because I want to modify the copy.
• Use SetTokenInformation to modify the Session ID in the copied token.
• Call CreateProcessAsUser to start my chosen app in the specified session.

Now, where am I going to get a session ID from?  There are a number of ways actually, but one option is to use the WTSEnumerateSessions function.

When I create a process in this way, the new process is a child of my highly-privileged service.  My service code has full control over that new process, and communicate with it in a variety of ways.  If the user in the target session doesn't have admin rights they won't be able to stop this process because I'm using my highly-privileged token (slightly modified) to launch it.

I'm sure you'll agree this is exactly the kind of operation that should require high privilege.  In my sample code above, the actual line that requires high privilege is that call to SetTokenInformation.  If you use that API to set the session ID, as I have, the calling process needs to have SeTcbPrivilege, which is otherwise known as "Act as part of the operating system".  So my service runs as LocalSystem.

You know the feature that Windows has had since XP, that allows "fast user switching"...?  I knew it was implemented using some variant of Terminal Services, but until recently I thought that different users' sessions were contained inside separate Window Stations (Winstas) and their applications would render on to separate Desktops.  Not so.  Here's what actually happens...

When Windows boots, you get a single interactive Winsta, whose name is Winsta0.  There is a single Desktop in that Winsta, whose name is Winlogon.  This is the desktop that displays the various login buttons, the power menu and accessibility options.  When you log in, as soon as the shell has something to display, Windows switches to another desktop whose name is Default.  That is where all your applications render their output.  If and when a screensaver is activated in your session, Windows will switch to a third desktop, solely used for screensavers.

• You will never have another Desktop or Winsta unless one of your applications explicitly creates one.
• The Winlogon and Screensaver desktops are "secure" - once they are showing you might have to enter a password to get back to the Default desktop.

So how is it that I can have multiple user sessions active on my machine, and UI belonging to each user's processes are somehow kept separate even though they all use the same Winsta and the same Desktop?  The key's in the access token.

Every process has an access token that represents the user account under which the process is running. Each thread inherits that token, against which the OS will check ACLs when the thread calls an API that attempts to access some secured resource.  So the access token is a runtime object that represents a user account.  But it also contains a member called SessionId, and that refers to the user session that the process should run it.  In turn, that determines which processes render their UI into the Default desktop when a specific user is active on the machine.

You can use the Process Explorer tool from www.sysinternals.com to inspect the access token of a running process.  Just right-click on a process in the list, choose Properties and click on the Security tab.

Armed with this knowledge, and a highly-privileged account, we can inject a process into somebody's already-established session.  In my next post I'll show how.

In recent years I've done quite a bit of performance and scalability testing of apps on the Microsoft platform.  I know plenty of people who are more knowledgeable about software performance than I am, but at the same time a little knowledge about how to measure and affect performance in software is a surprisingly rare commodity.  I guess it's a skill-set that not many people have the time and/or inclination to obtain.  Certainly, I only learned what I know because employers have put me in positions where I've had to learn that stuff.

I'm glad they did, because it's a fascinating area.  One of the things I like about this area is that so much of it appears counter-intuitive at first glance.  I'll give you some examples...

1.  Low CPU usage - good or bad?

Let's say we have some code running on an application server.  We test it by submitting a certain workload, and it takes a few minutes to work through it.  We observe the total CPU usage to be around 50%.  I've been in this situation countless times, and more often than not, people look at the CPU usage and say "great, our code is efficient, we have CPU to spare".  But let's look at that again: CPU 50%, and it takes a few minutes to work though the workload.  The right observation would be "why isn't my CPU running at (or near) 100%, and getting through the workload quicker?"

Perhaps because people are used to things slowing down when their desktop CPU creeps towards 100%, they assume that high CPU is bad.  On a dedicated application server, low CPU is bad, because it means that something is preventing that server CPU from working as hard as we'd like.  That is, we want the CPU to be working harder doing useful work.  Obviously there's no point increasing CPU work if our workload isn't processed any quicker.

So in our scenario, we can say that either (a) we're not generating the workload quickly enough to keep the CPU busy, or (b) the app is constrained by access to other resources, such as I/O on network or disk, or maybe contention around locking constructs.  Either way, we need to do some work to relieve that constraint and get the CPU working harder.

2. Asymmetric CPU usage

I'm sure I don't need to say why parallelism is important.  We in the software industry need to get much better at concurrent programming I think.  Most of us now have multiple CPU cores in our development machines.  That means, if we don't look in the right places, we might get a misleading view of the CPU usage in our applications.

It's surprisingly easy to build a .NET application that does most of its work on one specific thread.  This is particularly true of a Windows Forms desktop application.  I've seen apps that, although written in C#, were essentially VB6 apps because only the UI thread was used for doing any work.  On a dual-core machine, you might see something like this in task manager…

If you only ever look at an aggregated CPU usage value, you might be proud of the fact that your CPU usage never goes above 50% (although as I said in point 1 this is not necessarily something to be proud of).  In fact, this is showing a real problem in the design of your app, because no matter how you upgrade your hardware you're unlikely to make this app go any faster.

I guess I should admit that, right now, the app I'm working on looks a little bit like this.  Physician heal thyself, etc.

If you're using perfmon to analyse your app's CPU usage, you'll want to use the Process object so that you can view CPU usage just for your app's process(es).  It's worth pointing out that the CPU counters in the Process object go up to more than 100%.  If you have two CPU cores in the machine, it goes to 200%, or 400% for four cores, etc.

3. Extrapolation

One of the most interesting human factors around software performance is the common belief that software performance behaves linearly as variables change.  You most often see this as people trying to extrapolate performance figures to assume how software will perform on bigger hardware, with greater data volumes, or with more concurrent users, etc.  In general, it's safest to assume that you simply cannot extrapolate.

When you're working to improve the performance of an application, you find the first problem; let's say your app slows down to the point of being unusable when you're loading a large quantity of data into it.  Someone might say "ok, we'll work on getting that data loading problem fixed.  In the meantime, you find the next slowest part of the app and work on that".  The problem with this is that "the next slowest part of the app" might well be different when you manage to get that larger quantity of data loaded.  The slowest routine to operate on 100 items may not be the slowest routine to operate on 10000 items.

In many ways, improving the performance of an application is like mending a leaky garden hose.  You find the first problem and fix it, then you see the next problem and fix that, then the next, etc.

I have a Mac.  Nothing fancy, just a small MacBook.  I don't think it was a reaction to leaving Microsoft.  In fact, I dual-boot it with Windows Vista.  I need Vista because I am a Windows developer, after all.  Having said that, I have grown to like the Mac.  These days I'm doing most of my admin in the Mac partition, although I do use Microsoft Office:mac 2008.

Curiosity got the better of me, and I decided to try my hand at development on the Mac.  I should say that I was a C and C++ developer earlier in my career, which makes it easier for me to adapt to Objective-C Cocoa development than would be the case if I had only done .NET development.

There's a lot I could say by way of comparison between Mac Cocoa dev and .NET development for Windows.  Maybe I will say more about it at some point.  For now, let me just say that .NET developers have a much easier time of things than our Mac brethren.

Some of my time at Microsoft was spent in a marketing group, and one of the facts of life there is that, among all Microsoft's customers, developers are some of the harshest critics of the company.  All I can say is, some of those developers should try building apps for the Mac.  I think they might have a bit more love for Microsoft afterwards :-)

For a long time Microsoft has courted developers.  I think their focus on maintaining a productive development platform has helped their success in no small part.  Apple on the other hand seems to do almost nothing for developers.  Their online documentation is extremely dry compared with MSDN, and their development tools are idiosyncratic, to say the least.

So, .NET developers, consider those worse off than yourself :-)  And if Apple ever gets its act together and produces an online developer marketing channel* to rival MSDN then Microsoft will really have something to worry about.

I haven't given up on Mac development.  It's harder, but that's not necessarily bad.  It's always useful to have more strings to your bow.

*yes, MSDN is a marketing channel.  You knew that, didn't you?

I just saw one of the latest batch of Microsoft TV adverts (I'm in the UK by the way).  As usual, it's banal, and leaves me wondering what the heck it was trying to say.  For such a wealthy company, Microsoft has some shockingly-bad TV commercials.  When I worked there, I seem to remember that most of my colleagues agreed.

Look at the Mac vs. PC campaign run by Apple (which I guess this new Microsoft ad is playing on).  They were fun.  Enjoyable to watch.  They made a point.  People remembered them.  All this Microsoft ad ("I'm a PC") did is remind me of the Apple campaign and help cement Apple's message in my mind.

I currently spend a lot of time working on a development project with a client, and on that project we recently changed our source control system.  We were originally using ClearCase, and it's hard to be too critical of ClearCase.  I mean, it's been around for a long time, and it's still in use on some of the largest and most complex software projects.  It can be quite hard to get your head around the first time you use it, but if you're working on one of those large, complex software projects, I suppose you'd better be up to understanding ClearCase.

Anyway, for various reasons our project recently changed from ClearCase to Perforce.  From where I stand, these two products couldn't be more different : -

• ClearCase costs a lot of money (although I'm sure it's worth every penny) whereas Perforce has a sliding tariff starting from "free".
• In ClearCase the default working paradigm is that everyone has a private branch.  They can check in/out on that branch without affecting anyone else, and when happy they integrate back onto a shared branch.  Perforce supports the idea of branching but it's a less-frequently-used option.  The normal model is one of multiple concurrent checkouts; an idea which I've found a bit daunting in the past, but which seems to work very well in practice.

Both products have been totally reliable for me, in the sense that we haven't lost any work because of them.  I also feel like I'm being more productive with Perforce, mainly because in our environment a ClearCase integration operation could take quite a few minutes, whereas everything I do with Perforce is more-or-less instantaneous.

Because you can download and use (with some limitations) Perforce for free, I have it installed on my personal development systems as well.  I really like using the same tools everywhere.  Even my Mac OSX system has Perforce installed, and the client app looks/behaves the same everywhere.

Congratulations to Perforce.  I like your product, and I like your business model.  And they didn't even pay me to write this :-)

Recently in our app we were getting a System.IO.FileNotFoundException.  The trouble was, the exception detail didn't tell us what particular file was missing.  Our .NET app has a dependency on a COM DLL, which in turn has good, old-fashioned, dynamic-linkage dependencies on other DLLs.

Initially, I tried Process Monitor from sysinternals to log all filesystem access, but no joy there.  This clearly wasn't normal File I/O, it was a module load that was failing.  By the way, you can run those sysinternals utilities directly from http://live.sysinternals.com now, which is handy.

There might be a simple way to resolve this problem, but I couldn't think of it, so I resorted to a fairly obscure method, involving WinDbg.  When I was at Microsoft we used to joke that Microsoft really has only one tool, and WinDbg is it.  You can do everything in there.  It's not easy to get started with it, but I do recommend persevering.  It's best if you work near someone that already knows it, and get them to show you.

I guessed that the failure was occurring in a call to LoadLibrary or LoadLibraryEx, so I wanted to put a breakpoint on those functions and look at the string that was passed in - the name of the module to load.  Here are the commands I entered into WinDbg...

bp kernel32!LoadLibraryW "r $t0 = ebp+8; r $t1 = @@c++( *((long *)@$t0 )); dc $t1 L64; g"

bp kernel32!LoadLibraryExW "r $t0 = ebp+8; r $t1 = @@c++( *((long *)@$t0 )); dc $t1 L64; g"

Each command sets a breakpoint in the relevant function, and defines a command to run when the debugger hits that breakpoint.  In each case I find a pointer to the first parameter (which is at ebp+8), then I dereference this pointer and dump out 64 characters found at that address.  The final "g" is to make the debugger automatically continue from the breakpoint.

What did this achieve?  Well, running the app in the debugger dumped out all the names of the modules that were loaded by these two functions.  When the FileNotFoundException was thrown, I just had to look at the most recent filename dumped out and that was the module that failed to load.

Almost done, but not quite.  The name I just found is of the module that couldn't be loaded, but it could be found alright.  Let's call it "banana.dll".  It couldn't be loaded because it had a dependency on another DLL that couldn't be found.  To find out what that might be, I did a "dumpbin /imports banana.dll".  That told me what DLLs are imported by banana.dll.  I just had to look through that list to find the one that was missing from my filesystem.

This is a case of "blogging for posterity" in case I ever need to do this again.  Hopefully others can benefit from it too, though :-)

Finally, I have to question the wisdom of using FileNotFoundException to represent module load failures.  No wonder the exception detail couldn't tell me which file was missing.  Perhaps there should be a separate exception class for this?

Update: there is a simpler way :-)

A colleague just told me that Depends.exe can solve this problem.  Open your .exe with Depends.exe and use the Profile menu.  It hooks into the LoadLibrary (and other) calls and dumps out any failures.  That would have saved me some time...

When I look at the viewing statistics for my blog I see that last month was the biggest month ever, in spite of the fact that I didn't post much, and that I was on holiday for half of the month.  Actually, there was quite a big increase compared with the previous month.  This leads me to suppose that blog stats can only ever go up.

When I worked at Microsoft, "the management" collected bloggers' stats each month.  In fact, it was a key metric that our performance was measured on, and one that I never excelled in.  I don't think I'm giving away any secrets there.  Sure, when you read a blog that's from a Microsoft employee you can probably expect that blogging is a part of his/her job that they're measured on, but that doesn't make the blog less valuable.  Although there are a few worthless ones :-)

In general I think bloggers are seen as people donating their knowledge out of a spirit of generosity, and for a great many that is so, but you should always keep in mind that quite a few blogs are created and maintained just because the boss says so.

I'm sure other technology companies encourage blogging among their staff too.  This is not a "Microsoft is evil" post. Blogging has become a very important marketing channel for technology companies.

The good news is that most bloggers who do well at achieving high stats do so because their content is high quality and they keep it coming at a prodigious rate, whether or not their boss is keeping score.  But I must say I have known a couple of blogs that managed to achieve high numbers without seeming (to me) very worthwhile.  There are all kinds of tricks a wise blogger can carry out that will hugely inflate their stats, more quickly than generating a lot of useful content that a lot of people want.  Unfortunately, many employers can't see beyond the raw numbers.  Quantity beats quality every time.

When you're reading blogs, why not see if you can tell whether the blogger is playing a numbers game...?

I was just reading the lastest MSDN UK Flash newsletter, and was very interested in the survey results.  For the uninitiated, each issue of the Flash contains a survey that readers can respond to.  In the next issue, they show the results of that survey.  The most recent survey question was "What would interest you most around new versions of Visual Studio?" and the largest response (34%) was for "Stability, Reliability, Performance".  I find this quite surprising.

In the 8 years I worked at Microsoft, I went through several versions of Visual Studio, including pre-release builds a lot of the time, and I must say I always found its stability and reliability pretty good.  Performance was never a problem for me either.  This was also true in my life before Microsoft.

But in all those roles I actually installed Visual Studio myself, from a disc or from the network using Microsoft's supplied setup.exe.  In addition, I never had cause to install any non-Microsoft add-in for Visual Studio.

Recently I've been using Visual Studio on a machine managed by a customer of mine.  They use a 3rd-party packaging system to deploy everything to their managed desktops.  I don't know the details of how this works, but I'm guessing that it's some kind of registry/filesystem snapshot.

It's clear to anyone that's installed Visual Studio from a disc that the process is considerably more complex that dumping stuff straight into the registry and the filesystem.  Maybe it shouldn't be - I could happily support that view - but it's clear that it is more complex than that.  Otherwise why on earth would it take so long, and why would it display messages like "generating scripts"...?

So if we accept that VS is somehow "hard to install" we should probably believe that any 3rd-party packaging tool is unlikely to deduce all the correct logic to use when installing VS.  In fact, I find VS decidedly less stable on this managed desktop than I've ever found it to be in the past, on systems where I used Microsoft's setup.exe.

Also, this managed desktop has several add-ins for VS installed.  Occasionally VS has crashed and I've caught it in WinDbg, so I know it was at least one of the add-ins that caused the problem.  I guess that VS has become a platform in its own right and people are blaming it for faults in components that they've installed on top.

Microsoft used to get the blame for all blue screens in Windows but I think these days most users realise that it's more likely to be a 3rd-party device driver at fault.  Microsoft had to educate its customers by capturing data about their software failures and publishing data about the results.  Kind of like "naming and shaming" except I don't think they ever published the names of the culprits.

I wonder how many of the survey respondents were using some "odd" deployment tool, or some 3rd-party add-in.  Did they think about whose code was running when they experienced those stability issues?

So this survey result makes me think that perhaps the most useful features in future versions of VS would be to capture and publish data about which add-ins are crashing it, and to simplify the setup to the extent that a simple registry/filesystem snapshot is all you need to deploy it reliably.

One of the things I find most interesting about software development is the people who do it.  I think you can spot personality traits by reading people's code :-)  A few years ago, I was introduced to the Myers-Briggs Type Indicator, and I was interested that it seems to categorize people according to 4 axes.  I was a little dismissive of it at the time because, of course, a personality is more than 4-dimensional.  But as time has passed, and I've met more software developers, and read more of their code, I've started to realize that there are common traits, and there probably are ways to categorize developers.

I should say, this is just idle chat.  I am not suggesting that there's anything useful to be gained from this sort of analysis.  I just find it interesting.  In particular, I don't much care for the idea that an employer might try to assemble a team based on different types.  But some days, a difference between types of developer hits me right between the eyes, and today is one such day.

Have you ever read some code and found a comment like "was getting unexpected exceptions here, but with this Sleep(500) it works ok".  To my mind, the person who wrote this didn't feel in control of the machine.  It's like "one man and his dog".  I mean, the shepherd and his dog might consider themselves to be in control, but in reality the sheep have brains too.  Ultimately it's their decision whether they walk into the little pen.  I guess their thought process is something like "I might as well go where he wants, otherwise it's another 15 mins of being chased around the hillside".

But computers are even more stupid than sheep.  You can try to coax it into doing what you want with little things like Sleep(500), but it has no concept of "you've been debugging me for hours now, I'll just give in".  So coaxing and goading a computer into doing what you want is not an effective approach.  You need to grab hold of it and make it do your bidding.  Instead of the Sleep(500), when the exception happens, debug it.  Find out exactly what's happening and then stop it happening.  Master the machine.

So some developers are masters of the machine, while some are shepherds.  In truth, I don't think it's a binary choice, it's more like a spectrum; master at one end, shepherd at the other.  So...

M <------|------> S

...whereabouts would you put yourself on that spectrum?

I've been using SQL Server Compact Edition v3.5 for a little while now, and while the database engine has performed faultlessly so far, the same cannot be said of the management tool that's built into VS2008.  Perhaps that has improved with SP1 of VS2008, I haven't checked yet.

Today, I downloaded the release candidate of SQL Server 2008, just to get the management tools and see if I can connect to my SQLCE3.5 database from there.  I can, and it seems to work well.  I must say that I haven't spent long working with it yet, but it's great to have access to some of the features I'm familiar with from "normal" SQL Server, such as a graphical view of the execution plan.

Of course, it's still a release candidate.  My early optimism might soon wear off :-)

I'm currently working on an app that has to run on v2.0 of the .NET platform (with no SP applied).  The particular behaviour I'm talking about may have changed in more recent versions - I haven't had time to check - but I was struck by something and just had to write about it.

When you're working on a UserControl, Visual Studio sets the value of a property DesignMode to true whenever it loads your control into the visual designer.  This allows you to ensure that any code that depends on runtime artifacts doesn't run in the designer.  But if you search, you'll find lots of folks bemoaning the fact that DesignMode isn't set before the constructor of your control runs.  Again, maybe this has changed, I should check...

That means you have a problem if code in your constructor depends on constructs that are only available at runtime.  There are a number of workarounds posted on the web, but none of them struck me as very nice, and some just didn't work for me.  So I did the "obvious" workaround.  I took the code that couldn't run in the designer, moved it out of my constructor and into a separate function, and did a check for DesignMode there.  It works fine.

Now my control has a simpler constructor.  It just initialises fields with values that are always the same.  All the logic that calls into other parts of the app has been moved into my new function.  I feel that my control is somehow "less-coupled" to other parts of the app now that I can construct it without those things being there.  It feels more correct.

What's the catch?  Well, something has to call my new function.  That could be the code that instantiates my control, or I could call it from my own control's Load event handler, or from some other event handler inside my control.  In my case, I chose to let the calling code do it.  it worked fine but I still walked away thinking I'd made my codebase more complex to workaround an "issue" in VS2005.

But today I was doing further development against my control.  It struck me that the additional function I introduced has actually allowed me greater freedom in the way I initialize the control and has actually simplified my codebase in the end.  The feeling of correctness I got from refactoring the constructor was borne out in my ability to reuse this control in ways I didn't anticipate.

There's a lesson in that.  You might feel coerced into writing code a certain way, just to workaround a quirk in your tools or platform, but that doesn't mean the workaround is actually worse than what you originally intended.  It could be better.

I've started a one-man campaign. In fact I started it years ago when .NET obfuscation first appeared. I don't like any of the .NET obfuscation technologies. I don't think obfuscation is worth bothering with, and it makes the developer's life just a little bit harder. So don't do it.

Think about it...

1. Obfuscation is not encryption. That's really important.

2. People being able to see your code is not a bad thing most of the time.

3. If you have a case where it's really necessary to hide your code then, by definition, obfuscation isn't good enough. You need to encrypt that really sensitive piece of code, perhaps using something like Microsoft's SLPS.  Even then, only encrypt the sensitive stuff, not the whole app.

4. .NET didn't change anything. Native code is not encrypted either. If someone can benefit from reverse-engineering your code, they are just as likely to do that to a native-code app as for a .NET app.

5. Obfuscation might deter the very casual observer, but how many casual observers fire up ildasm or Reflector? Honestly? And how much harm do you think casual observers will do to your business?

6. The ability to run ildasm and Reflector over my code is very useful to me, and makes me more productive.

7. Obfuscation might give your company a feeling of protection that it really doesn't offer.

8. No matter what the obfuscation-tool vendors do, the debug process is always slightly more complicated for an obfuscated app. If there was some value in obfuscating, this might be ok. But there just isn't.