The Day Microsoft Became Discriminatory

Let me show you a time when Microsoft, or maybe Microsoft Philippines only, discriminated against Firefox users, or maybe other browser users as well. If you look closely, you can observe that msevents.ph showed a link to the event details on IE but removed it from Firefox.





Talk about blatant web discrimination. It's like racial discrimination for the 21st century. I hope Microsoft renews its stand from this kind of discrimination. It's OK to be discriminate about browser-specific features (this javascript works in this or that browser only), but not in true content information. *sigh*

Edit: Oh, OK. As of this edit, they have now featured a textual link instead of the graphical link. Good enough! :-D

Allow me to show you scenes from our recent biking tour with UCAP or
United Cycling Associations of the Philippines. Our route traced major thoroughfares of Quezon City measuring around 44 kilometers.
At a brisk pace with moderate to extreme inclines and declines, the ride took place at less than 2 hours from 8 AM to 10 AM.

We just came from Palawan this past weekend. I was lucky enough to accompany my family and in-laws in this trip. For a detailed photo blog, see http://photos.yahoo.com/loloyd/. Ahh... 

How I Defeated A BSOD in Windows XP
September 27, 2005, One Fateful Tuesday

(BSOD = Blue Screen Of Death)

The good news about the recent bandwidth speed upgrade quickly spread around the IT department where I work. After finishing up with a couple of tasks, I decided to take advantage of the situation by instituting automatic updates for my Windows XP Pro (OEM) that morning. I already had Service Pack 2 (SP2) v.2096 (RC1 or Release Candidate 1) installed. For some godforsaken Microsoft-vodoo reason, my system, sitting on a 1.8 GHz Pentium 4 with 256 MB RAM, would operate ABNORMALLY when the RTM final version of Service Pack 2 was installed in it. So, since the third quarter of last year (2004), I really had to settle with only SP2 RC1 just to keep annoyances to minimum manageable levels. So I right-clicked on My Computer, brought up its Properties, and under the Automatic Updates tab in the resulting dialog box, I clicked on the Windows Update Web site link.

As I was about to eat lunch (oh, btw, did I mention that I do my lunch right in front of my PC?), the system tray popped up a balloon telling me that around 27 or so updates have been downloaded and are ready to be installed. I browsed through these updates - many were flagged as critical security issues, and then there were also hotfixes. I relented and allowed ALL updates to be installed. What could possibly go wrong anyway? Or so, I thought. A few moments later a system tray icon popped up again telling me I needed to reboot so that the updates can be fully installed. OK. That was maybe before my fifth spoonful of lunch.

The machine rebooted. Out came the Windows XP startup logo.

Then it suddenly rebooted again. The Windows XP startup logo displayed once more.

And it rebooted itself again. Then the same Windows XP startup logo showed up. A blue flicker abruptly flashed before the system rebooted itself once again. Panic did not creep in. But disgust and anger did. I let the machine reboot itself a couple of times more, crazily thinking that maybe, just maybe, the number of times it needed to reboot matched the number of updates it had installed. I was sarcastically kidding myself. Interestingly, all the various safe mode startup options, including the Last Known Good Config and Debug modes, failed to get past the problem. What should I do?

I'm no newbie to OS troubleshooting. I first had a hand at it back in the days of console-based DOS. Around here, we call text-only modes as "console-based". Back then, DOS was simple and things were pretty straightforward. I also wasted considerable hours and days tinkering, fixing, tweaking and hacking 16-bit Windows 3.x. Then came 32-bit Windows 95 and 98. And so on. I remember wasting a lot of time agonizing over and over just to troubleshoot simple things, just to get things back to the way they were, without wasting hard disk space resources and suffering performance compromises. I consider myself a relatively very careful person, that's why I can do away without using System Restore and such tools. My antivirus apps are non-TSRs either (stopped services) as I only use them for passive file-level scanning. And I really get closely acquainted with my task processes list, even with many of the drivers and kernel hooks. That's how I get things running smoothly and swiftly.

The old memories of pain, anguish and agony over troubleshooting OS problems hauntingly flashed back. I had more than a couple of options, first was to try to restore everything back to normal manually. The second is to salvage my old files from the system drive and then do a reformat, then do a clean reinstall... of Windows XP and all of the software that I use. This was very unappealing since I have more than 50 pieces of software that I quite actively use. The third and unlikely option was just to reinstall Windows XP over the damaged one. That was also unappealing because I'd rather have a clean install than a washed up/over one. Besides, I would have to reinstall everything too if I chose option number 3. Figuring that I would need around a week's time of reinstalling if I did either options 2 and 3, I decided to have a go at option 1 first. I imposed a deadline for option 1 to work for the rest of the day. If I had not been able to successfully fix things come 5 or 6 PM, or at the end of the working day, I'd have to resign to option 2 for the next day.

Borrowing a colleague's PC, I surfed Microsoft's website for some information about the computer "restarting repeatedly". I got the page information entitled "Black startup screen is briefly displayed and your computer restarts repeatedly when you try to start Windows XP", article ID 314466 <http://support.microsoft.com/?scid=kb;en-us;314466&spid=1173&sid=182>. It discussed a way on how to stop the computer from restarting repeatedly, so that I could capture the error message on the blue screen of death (BSOD), and then work things around from thereon. In essence, what I needed to do was to edit the registry of the damaged Windows XP to stop it from rebooting automatically on a failed boot basis.

I couldn't follow the first method in the article even with the Windows XP on hand for the Recovery Console. Now, the second method I did follow. It instructed me to install Windows XP to a different folder and tinker with the old registry using the registry editor of the fresh one. OK. Luckily I have 9 partitions across 3 hard drives. So I chose to install the new Windows XP repair version on drive D leaving my old but damaged Windows XP in drive C intact. Installing Windows XP on drive D took about 30 minutes. Once I booted using that repair mode Windows XP (I dub thee henceforth "repair mode Windows XP"), I opened regedit, and loaded the hive containing the CrashControl key from my damaged Windows XP following method 2 as described in the article. That solved the repetitive auto-restarting problem part.

Booting the old Windows XP, I finally gathered this BSOD information (text may not be fully verbatim; and capitalization and spacing may not have been fully consistent with the actual BSOD message):

===
STOP: c0000269 {Illegal System DLL Relocation}
The system DLL user32.dll was was relocated in memory.
The application will not run properly. The relocation occurred
because the DLL c:\windows\system32\ntdll.dll occupied an address
range reserved for Windows system DLLs. The vendor supplying
the DLL should %s %s
===

I now had clues on what DLLs caused my system to foul up. Searching the internet about "c0000269" and "Illegal System DLL Relocation" using Google practically produced nothing of much use. I was quite interested in the Thai version of Microsoft's knowledge base though <http://www.microsoft.com/thailand/security/nt4thai.asp>, but it looked like it only applied to Windows NT. And I couldn't possibly make out what the heck it was saying with all the Thai character text all over it. Systran, the internet translation experts <http://www.systransoft.com/> and <http://www.tranexp.com/>, failed to provide a web page translator from Thai to English. At least I was not the only one who suffered in this problem. But there I still was, practically left to my own devices.

At that time, I began to formulate my own step-wise objective, hoping for it all to lead me to the end objective - which was to get everything back to normal or to the way it was before I installed the Windows updates that destroyed the functional operability of my Windows XP. My first step-wise objective was to manually recover the old working versions of user32.dll or ntdll.dll or both. Whichever works would be fine for me. Using the repair mode Windows XP version on drive D, I was able to browse around my old C:\WINDOWS directory by bringing up a Command Console (run cmd.exe). Using a lot of the following directives helped me determine which files to experiment on:

===
c:
cd windows
dir ntdll*.dll /s /a
dir user32*.dll /s /a
dir /ad /od
===

I used [dir ntdll*.dll /s /a] to get all possible ntdll.dll replacement versions. I used [dir user32*.dll /s /a] to achieve a similar result for the file user32.dll. Then I used [dir /ad /od] under C:\WINDOWS to get a list of recently created subdirectories that may have hopefully kept older versions of the replaced files.

There were actually a few version instances of ntdll.dll and user32.dll sitting in various subdirectories inside C:\WINDOWS. The only files that actually mattered were those inside C:\WINDOWS\SYSTEM32. All others (inside C:\WINDOWS\ServicePackFiles\i386 and C:\WINDOWS\$NtServicePackUninstall$) were either backups or obsoleted versions. There were also version instances of the same files in directories similar to C:\WINDOWS\$NtUninstallQ815021$ and C:\WINDOWS\$NtUninstallKB824141$ which may mean obsoleted versions caused by the update discussed in the Microsoft Knowledge Base with the articles identified as Q815021 and KB824141 respectively. I simply disregarded the files under C:\WINDOWS\SoftwareDistribution\Download.

===
03/12/2004 02:18 AM 722,944 ntdll.dll
...
08/29/2002 08:00 PM 668,672 ntdll.dll
...
05/01/2003 04:56 PM 654,336 ntdll.dll
...
03/12/2004 02:18 AM 578,048 user32.dll
...
03/03/2005 02:09 AM 577,024 user32.dll
...
etc
===

I did not know if either ntdll.dll or user32.dll was at fault. So basically, what I did was to rename both ntdll.dll and user32.dll in the C:\WINDOWS\SYSTEM32 to something else - my method was to extend their filenames to their matching time stamp, i.e.:
03/12/2004 02:18 AM 722,944 ntdll.dll
would become
03/12/2004 02:18 AM 722,944 ntdll_200403120218.dll
and
03/03/2005 02:09 AM 577,024 user32.dll
would become
03/03/2005 02:09 AM 577,024 user32_200503030209.dll

I copied various versions of ntdll.dll from backup directories with unmatched time stamps to C:\WINDOWS\SYSTEM32 then tried rebooting on that Windows XP. It failed. BSOD still appeared on startup but with a different error message this time. I also did the same thing to user32.dll and a similar BSOD error message resulted on bootup. I tried various combinations of user32.dll and ntdll.dll instance versions at the same time and rebooting each time with no luck - BSOD on startup still. There may be some other files that needed replacing. That's where
[dir /ad /od]
came in handy.

The directory listing sorted by date inside C:\WINDOWS showed KB-related updates. One subdirectory was $hf_mig$ while the others began with $NtUninstallKB or $NtUninstallQ followed by some numerical ID. Since this list was sorted by date I inspected the insides of each subdirectory timestamped September 27, 2005. I noticed that most of the subdirectories each had a subdirectory named spuninst inside. And inside this subdirectory, a file named spuninst.bat (or spuninst.txt) details a possible way on how to back track to previous copies of replaced files during an update installation. Illustratively, the following subdirectory entry
08/19/2004 10:41 PM $NtUninstallKB838907$
inside C:\WINDOWS would contain the file C:\WINDOWS\$NtUninstallKB838907$\spuninst\spuninst.txt. This particular file would contain uninstallation procedures for that particular kind of update (on KB838907 for this instance) and to where the replacement files were originally placed.

So I browsed through all such $NtUninstallKB and $NtUninstallQ subdirectories and realized that, in some of those subdirectories, user32.dll came with other files as well such as:
===
authz.dll
ntkrnlmp.exe
ntkrnlpa.exe
ntoskrnl.exe
win32k.sys
winsrv.dll
...
===

I don't remember exactly what other files were there but they conveniently came in one whole group as described in the spuninst.txt (or spuninst.bat) file. There were around 3 bundles of updates that included user32.dll so I had to try each bundle group at a time (from dated latest to earliest) with reboot. LO AND BEHOLD, ALLELUIA, the earliest bundle of DLLs with the directory that began with C:\WINDOWS\$NtUninstall directory date stamped September 27, 2005 allowed my old Windows to finally boot up!!! Hooraaayy!!! Yeepee!!! I almost did the dance of joy as popularized by the old sitcom Perfect Strangers. In summary, what worked for my case was this combination of files:

===
03/12/2004 02:18 AM 56,832 authz.dll
03/12/2004 02:18 AM 722,944 ntdll.dll
03/02/2005 09:02 AM 2,135,552 ntkrnlmp.exe
03/12/2004 12:44 AM 2,069,888 ntkrnlpa.exe
03/12/2004 02:18 AM 578,048 user32.dll
03/12/2004 01:12 AM 1,829,120 win32k.sys
03/12/2004 02:18 AM 291,840 winsrv.dll
...
===

I don't know if I have failed to mention a file or so. But one thing for sure is that I made my replacements by grouped bundle.

Although I was able to successfully boot up my old Windows XP, the old "activate Windows XP" message initially kept popping up, reminding me to activate my copy of Windows XP within the next 3 days. I have already activated this Windows XP version around 3 years before but I guess this is just a side effect of all the mess that has happened. Also, explorer.exe kept failing due to some illegal function call error. So there I was, with another problem to solve. I successfully booted but I had no desktop, no system tray, no taskbar and no Start menu. I suspected that this was due to some incompatibilities with the manual remedy that I just had made. So I pressed Ctrl+Alt+Del to bring up the Windows Task Manager. Using its File|Run... menu, I was able to bring up the Add/Remove Programs control panel applet (run appwiz.cpl, it is inside C:\WINDOWS\System32). I marked the "Show updates" check box at the top and scrolled down to Windows XP - Software Updates. My next step-wise objective was then to remove the updates installed during that day - September 27, 2005.

I systematically uninstalled each update that was installed on September 27, 2005 (flagged at the right side of the list). Some updates needed to be removed in the correct sequence. To figure out the sequence, I patiently tried to remove each update to see which update removed without any complaint. If there was a complaint, I moved to try to remove another update. Whenever asked to reboot mandatorily, I did so. I just persistently kept summoning up Task Manager and AppWiz.CPL for each reboot to remove the pesky updates installed that day. On one reboot, the error on explorer.exe ceased but a new one on ntdll.dll emerged. However, after I uninstalled the last piece of update, all bootup errors CEASED to exist. On one my last reboots, however, the system informed me that, due to the changes in my hardware (there was actually none at all), I had to re-activate my copy of Windows XP. It also informed me that many of my controller drivers were missing: SiS AGP controller, Monitor driver, Multimedia controller, USB controller, etc. This was easily fixed however as I just watched Windows XP correct itself and installed all the necessary drivers all by itself. Self-healing back at last. This was all done while my Windows XP SP1 installer CD was inserted in the CD bay. What's left in my updates list were update installations prior to September 27, 2005. I can't complain anymore.

All the abovementioned processes took place in 4 to 5 hours. Precious time wasted on troubleshooting, but I'm extremely happy with the outcome. It's better that than spending an entire week rebuilding the system (plus trying to find where some of the installer CDs have been misplaced hehehe).

At long last, my old Windows XP was back to normal operating conditions. My system lived peacefully long without those updates anyway. I believe my Windows XP firewall, free Sygate Personal Firewall and passive-mode Grisoft AVG antivirus are enough to keep viruses and trojan nuisances out of the way. That, and a mighty acquaintanceship and understanding of the tasks listed in my Task Manager, and drivers and kernel hooks in System Information (run msinfo32), under the Software Environment node. One can also check the startup processes there. Great for checking out digitally signed stuff.

That's basically how I defeated a BSOD in Windows XP.

===
Professional Corporate Executive Summary:
1 Installed around more than 27 critical updates and hotfixes before noontime of September 27, 2005, Tuesday.
2 Repetitive reboot on Windows XP startup logo realized afterwards.
3 Researched on repetitive reboot problem on the internet.
4 Fixed repetitive reboot by installing a fresh repair-mode Windows XP on a different drive and changing a registry setting from the old Windows XP.
5 Logged blue screen error message on bootup of old Windows XP and researched on it.
6 Listed the files suspectedly necessary to replace or restore and restored them from backup copies. Reboot each time for each combination of files. In this case, ntdll.dll was not at fault. It was only user32.dll and its buddy files as listed above.
7 Succesfully rebooted after replacing user32.dll and its buddies with the oldest backup on that date, September 27, 2005. The files were timestamped differently so don't get confused there. The backup date is stamped ON THE SUBDIRECTORY.
8 Errors on explorer.exe and ntdll.dll were eventually realized after booting up. There was no desktop nor taskbar. Used Task Manager and AppWiz.CPL to uninstall each update that was installed on September 27, 2005.
9 Errors on bootup soon faded and the desktop returned to normal. However, drivers to many of the system controllers became missing. Windows XP successfully healed itself even without fingers crossed. Cool.
10 Re-activation of Windows XP was required. Done in a Snap with a capital S.
11 Everything fine and back to normal once again. Well ok, maybe also with some Luck with a capital L.
===

I hope I have helped a small part of the internet community by describing my problem and solutions above. For me, it's such a waste of time trying to rebuild/reinstall your software system practically from the ground up. Remember, when something bad happens with your Windows XP, don't panic, just get angry. Hehehe. My anger drove me to find a truly viable and actual working solution, and to write this blog. Email me at loloyd at loloyd.com (written that way to avoid spam-bots) for your comments and similar concerns. I do not, however, guarantee replies. Thanks for taking time reading this article.