Friday, June 1, 2007

XPe tip #50: Smallest Windows XP(e) based thin client image

Another post in the newsgroup has led me to an investigation in results of which I came up with probably the smallest networked XPe image that include fully functioning RDP client.

Basically I suggested to start with image that would include the following:
- Your hardware + HAL (only hardware you plan to support in the image)

- NTFS FS/File format. The latter is zero-size and may bring more components in as dependencies - however not actually required if you don't plan to format disks in NTFS under XPe.
NTFS will help you to compress the file system and thus to minimize the disk space usage.

- Basic: NT Loader/Nt Detect/Windows API - GDI, Kernel, User, Advanced/Shell32 API/Ole32, OleAut32/Language support/Winsock/Common Controls and dialogs/etc.

- Terminal Server Client

- Windows RAM Disk Driver (this is for RAM or Remote Boot support if you plan to use it)

- Winlogon or Minlogon (big difference in image footprint and significant difference in boot time)

- CMD shell (useful for debugging purposes). You can remove this shell later when you are done with the image config, or replace it with your own custom shell or the MSTSC.exe (RDP client) running as the shell.

If you do the above with Minlogon your image size can be less than 96Mb (uncompressed). With Winlogon it will grow by ~20-30Mb. The given numbers are rough and I got those off some of my demo images I have here with RDP client working. The actual image size may very dependent on some other system specs. I often do more optimizations for image sizes in TD and post-FBA even if Minlogon used. Then 96Mb can go down to 64Mb or less (with RDP still working).

Perhaps, the most sensitive area is the hardware support. If you use TAP output to create platform macro component please make sure to disable there as many component dependencies as possible based on your final target device specs. I typically use Selector Prototype component as the prototype for all my platform macros and it helps me to disable unnecessary hardware support in TD.

Often, the challenge is to get the networking to work properly in the MS Client network environment and not blow the image size with lots of network components that have over-crossing dependency chains.

So, continuing the path I just drew I got my VPC Minlogon image with fully capable RDP client running and tested down to uncompressed size ~38Mb / compressed ~28Mb.
Plus you'd want to add 2-3 more Mb for the Terminal Services dynamic cache that is created at run time.
Note: one real device the image would be smaller.

As you can plainly see, the image is small enough to fit on cheapest 32Mb USB stick (yes, possible to make it to boot the image off a USB 2.0 device). Just imagine - you carry a stick that you can boot off on any Intel/AMD based machine and it will give you a Terminal Server Client capabilities to your network - cool!

XPe tip #49: Why HORM restricted to EWF-RAM?

There was an interesting question posted to the newsgroup recently that I attempted to answer. Thought I want to add the ideas posted to my blog as well.

The question was why HORM (feature of XPe SP2) is limited to EWF-RAM mode. HORM stands for Hibernate-Once-Resume-Many embedded enabling feature.
In Fp2007 to the ewfmgr console tool (as well as to EWF API) a couple of new commands/functions have been added that are directly related to the HORM and basically allow you to activate and deactivate the feature at run time.

You will also find a lot of internals on how HORM works in this thread where Slobodan and me discussed the HORM feature implementations way before it was called HORM :-)

Now back to the original question. Why we are limitted to RAM overlay when HORM is used?
I guess the main "limitation" there of EWF Disk mode vs EWF RAM Reg is that by default the overlay data is "persistent" in Disk mode.
The major reason for using EWF RAM for HORM scenarios was that the changes you do while you work with the image that are redirected to RAM are going to get lost on reboot. This way whatever OS does about the hibernation file (hiberfil.sys) will be lost as well and you can always go back to the original state of the device - this is basically the main purpose of the HORM.
With EWF Disk the changes are redirected to Disk overlay (another hidden partition on the disk) and EWF will pick them up at the next boot until you clean the overlay or commit the changes (there is also a minor difference in behavior depending on what restore level you selected for the disk mode but it is irrelevant to this discussion).

So, imagine the following scenario:

1) HORM is enabled. Hibernation file holds a valid state of the system saved at some point of time.

2) EWF is enabled too. Disk overlay is used. Initially the cache is *clean* (not really possible with EWF Disk mode when you enable HORM but just for the sake of this explanation).

3) You boot the image and it boots fast from the hiberfil.sys file.

4) You work. FS (actually disk level) writes are redirected to Disk overlay, no files on the protected partition are actually changed on the disk including the hiberfil.sys.

5) You reboot the system (gracefully or not doesn't matter since you are EWF protected).

6) The system boots from the same (old!) hiberfil.sys file. This means it will restore all the driver and application states to whatever states they had in memory at the moment the golden image was hibernated. EWF driver also has some (old) state there where it remembers what was the Disk overlay content at that time, i.e. it thinks the overlay is *clean* (see step#2).
However, your actual Disk Overlay is not clean since it was modified in an earlier OS session (see step#4). Now you get a discrepancy between the disk overlay and its state in RAM (EWF state machine). This will likely lead to an exception within the EWF driver and, since EWF is a kernel driver, to a BSOD.

The situation is even worse if we go real and don't assume the *clean* initial state of the EWF overlay. Since at the time we hibernated the image the EWF was already enabled (it must be enabled) you already got some changes to the disk (often unintentional such as logs, registry hive changes, etc.) that were redirected to the overlay.


Another issue could be that if you need to commit the changes (and stop the HORM) you won't be able to do that. The actual commit in Disk overlay mode occurs at the next boot time after you issued the commit command. At boot EWF driver reads all the changes off the hidden data partition and applies them to the disk. Obviously, this happens way later than the OS (the loader and the kernel) loads the RAM image off the hiberfil.sys (i.e. restores the system from the hibernation file). Since the hiberfil.sys file is the old one the EWF driver there (its state machine) doesn't know the commit command was issued and you will never get out of the loop.


The only way to test the above scenario is to use one of the "hacks" we discussed with Slobodan in the newsgroup (see the link above). Otherwise, HORM will be disabled as reported by the ewfmgr. I also discovered that hibernation won't work at all while EWF Disk Mode is protecting (enabled) the system partition that holds the hiberfil.sys file.

The actual bug, in my opinion, is there in the latest Enhanced Write Filter component UI in TD. There Microsoft added the checkbox for HORM - very nice feature - but hesitate to disable the option when the settings are changed to Disk Overlay usage.

XPe tip #48: Get rid of System Tray icons and remap the UI

There are some 3rd party applications (and even Microsoft application like BlueTooth systray agent) that install icons in the SysTray area - the right-lower corner of the Desktop screen. Some of these application rely on the presence of that icon to provide a way to access their UI. Unfortunately, often there is no other way left.

When you are trying to get rid of Explorer shell (on some embedded devices that run a custom shell apps it will make sense) you will remove the SysTray ("Notification Area") as well since it is a part of the Explorer application. More precisely it is a part of the TaskBar feature of Explorer Desktop that runs as system shell.

So, the question is how to get rid of the SysTray (the Explorer) but yet have a way to access such application UI? No easy answer.

Here is what I'd suggest (posted my thoughts in this newsgroup thread).

1) The best approach would be to implement the missing Explorer features within your own custom shell application. It is really hard with regards to some features and requires a lot of debugging on XP/XPe to understand how those features work and how to invoke and integrate them into your own application.
Just to encourage you, I've done this for some of the Explorer features such as AutoPlay, Taskbar, Volume Manager and etc.

2) How 3rd party applications install the icons in the System Tray area?
It is basically about about one API function - Shell_NotifyIcon - that is being used by applications to add icons to SysTray area and be able to receive notifications from the shell by user clicks on those icons through the application window procedures (when you call that API you register your window procedure to receive the events). The Shell_NotifyIcon is exported by shell32.dll. Assuming you don't use shell32.dll in your image that is supposed to run a custom shell, you can create your own version of shell32.dll that will expose (export) that function only. Used as a shim it will give you the full control over what and when that application is doing in the UI.

3) Another way, much simpler, is to find out what messages are sent to the application window procedure when user clicks on the application specific icon in SysTray area or selects an item on the appropriate menu shown by the left-[right-]click on the SysTray icon. Knowing those messages and their parameters (likely WM_COMMAND messages) you can always implement them and send them out from your own shell to the 3rd party application being investigated. This way you can "remap" all the functions of the application icon in SysTray to your own UI.
The best way to watch for the messages is to start with Microsoft Spy++ tool. Next steps is to subclass the window (via global hooks in Win32) and watch for the messages through debug output.

XPe tip #47: UNMOUNTABLE_BOOT_DISK BSOD on VPC

While working on demo images that I deploy on VPC (Virtual PC 2004-2007) I noticed an interesting thing that makes a perfect sense though.

I use the newest feature of Virtual Server 2005 R2 (and I love the feature) - Virtual Disk Mounting. That basically requires Microsoft Virtual Server SCSI disk device driver.
What I noticed is that if I leave the VHD (virtual disk image) mounted in the OS on my development machine and I try to boot that image in VPC - I catch the UNMOUNTABLE_BOOT_DISK BSOD right away.

This makes sense since the volume is not available for writes from within the VPC as the host OS locks the VHD image with the new disk device driver in the chain.

The tip is simple - don't do that! Don't mount the VHD and try to boot from it in VPC.

XPe tip #46: Another EWF related BSOD: UNMOUNTABLE_BOOT_DISK

Just discovered that if you use EWF Disk Overlay mode and set the disk overlay size (in TD) to something very small (by default it is only 1024Kb) you are going to get the UNMOUNTABLE_BOOT_DISK BSOD (Blue Screen of Death) right after the second reboot after you enabled the EWF.

The reason is, probably, that when you enable EWF and reboot (to take the change in affect) the OS starts redirecting all the disk writes to the Disk overlay. It is trying to but obviously the size of the overlay (the size of the hidden partition that holds that Disk overlay) is not enough to cover all the changes, even unintentional ones like underlying system logs, changes to registry hives, etc. Somehow EWF doesn't give you any notifications about it (and that may make sense on a headless embedded devices, btw). At the next boot the disk writes saved in the overlay are validated and this leads to the BSOD.

It is actually easy to prove that it is the EWF (not corrupted FS or etc.) that is the cause fot the BSOD. Just open up the system registry hive of the crashed image offline (from another OS load such as XP Pro or WinPE) and remove the EWF from the list of volume upper filters for the disk driver. You will be able to boot the "patched" image just fine.

Another way to prove that is to delete the EWF hidden partition that holds the broken overlay. Obviously, EWF will become useless after that but you can always get it back to the default working state by issuing the following command "rundll32 ewfdll.dll, ConfigureEwf" that will re-create and initialize the EWF config (overlay) partition on the disk.

Thursday, May 31, 2007

XPe tip #45: EWF and Cloning

I guess the topic has been discussed in the newsgroup quite many times but since I just happened to run into a related issue and fixed the runtime I thought I should cover this in some details on this blog.

What if you are tasked to create an embedded image that must include EWF and will be deployed to devices as is. In other words, there won't be technicians who can fix known issues with the image deployment in field.

I am only going to cover here the case where all the target devices are identical - the same type of the hardware and number and type of peripherals attached. If this is not the case, it should be adopted/modified dependning on what hardware modifications you plan to do in the field (such as changing partition setup and so on).

The task first seem to be trivial - you just have to properly clone the image on multiple target devices. Dependening on how you plan the clonig process (factory mode, etc.) and what's the target working environment will be (LAN, domain network, Internet, etc.) you will or will not include and setup the System Cloning Tool component.

Unfortunately, that is only going to be that easy if you use EWF RAM Reg mode. The reason is simple - in RAM Reg mode EWF stores all the configuration settings in registry. So when you copy the image to another device, all the EWF settings will be properly copied along with that. No need to worry about hidden partitions and such. You can simply use xcopy to copy the image files. Certainly, cloning tools such as Ghost or Altiris can be used as well no problem.

However, in you happened to use EWF RAM or Disk mode you got yourself in troubles. If you just xcopy the image (or use ghost tools the same way you'd use to capture as smallest image as possible) at the cloned system launch you will notice that EWF is not working. If you execute EwfMgr commands it will report the it cannot find the EWF volume (and there it will refer to the EWF config volume that stores all the EWF configuration settings and, in case of EWF Disk mode, the overlay data). Obviosly, with the Windows standard tools (xcopy and etc.) you won't be even able to see the EWF partition. So another approach shuld be taken and here is a few I can think of:

1) Use Ghost's -IR switch (or similar switch of another cloning tool if suported) when capturing the golden image. It forces the ghost to grab every raw data byte of the source disk vs going through the file system to copy only files as the ghost without IR switch does.
Big disadvanges of this solution is that depending on the target device's hard disk setup you may end up with a GHO file with a huge size (ghost with IR switch doesn't know anything about files on the disk and capturing every single byte of the specified storage device). Also, this way you won't be able to operate and clone partitions, only entire disks.

2) Create a tool that will copy over the hidden EWF partition to all the target devices. Something like WinHEX editor or any partition management tool that allows you to view and save the content of disk sectors would help you here. I typically use Paragon Partition Manager for that.
This approach is quite complicated and not easy to automate without a 3rd party tool which may add some extra licensing to your device production.

3) Have EWF disabled in the golden image and re-enable it on the cloned runtime very early at the first boot. You can then xcopy the image files to target devices or use Ghost the regular way (FS aware).
However, this would only work if you re-create the EWF Config partition on the cloned image. Here is the command that will help you to accomplish that:
rundll32 ewfdll.dll, ConfigureEwf
- or -
rundll32 ewfdll.dll, ConfigureEwf Start
(both seem to have worked for me)

You can launch this command with a FBA Generic command set up to run after the cloning phase (phase number > 12000) or use [HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce] key or Startup Menu item if the Explorer Shell is used in the runtime. Otherwise you can use [HKLM\Software\Microsoft\Windows NT\CurrentVersion\Winlogon],"UserInit" key to add more command you want to run at run time. Just make sure to clean up that key and return its value to original (default) when the job is done. If you do all the commands from a batch file that will also add some .reg files that will do it.

Obviously, after you created the EWF Config partition you will need to issue "ewfmgr : -enable" command and reboot the target device so that EWF can take that command in affect. Again, all these commands are easy to combine in one batch script file.

4) Switch to use EWF RAM Reg mode. After all, there is almost no advantages of EWF RAM mode over the RAM Reg mode (except the ability to use -disable command). However, EWF with Disk Overlay has its own value that would be hard to replace with EWF RAM.

Friday, May 25, 2007

XPe tip #44: Where are the system shurtcuts?

Where the system shortcuts are coming from? Administrative Tools, Programs, Programs\Accessories (Accessibility, Communications, System Tools, Entertainment), Games, Messenger, Movie Maker, Wireless Network Setup Wizard, Remote Assistance, Windows Media Player?
The answer is trivial - "Standard Start Menu Shortcuts"component.

Btw, that component is a great resource for those who wants to learn how to create shortcuts on XPe directly in TD.

XPe tip #43: CDFS/UDFS

Problems accessing the content of CD or DVD under XPe keep being posted in the newsgroup.

The tip here is very simple - include the CDFS/UDFS components - CD/DVD file systems.
This is surely going to work assuming you've got all the CD-ROM/DVD-ROM driver issues resolved first.

XPe tip #42: Media file metadata access

This interesting post in the newsgroup led me to discovering another nasty bug of latest XP Embedded bits (SP2, FP2007).

Basically the issue was that the poster was having troubles accessing the metadata of media files on an NTFS partition. More specifically, their script code that called to getDetailsOf method of the Shell.Application object was failing to return proper results but returned the success error code.

The issue showed up when the developer moved to FP2007. First thought was the problem was there due to a missing component that could disappear (vs old builds) because of the major re-factoring Microsoft did in that Feature Pack.

However, the source of the problem was in the new component that was added to the database update - "Windows Media Player 10.0". The previous player components - "Windows Media Player 8.0" and "Windows Media Player 9.0" (this one is ghost component) - included the shmedia.dll library but the Player 10 didn't. That library is actually responsible for parsing file content, that shell API are loading up, and extracting and parsing the metadata that file may contain. That metadata is heavily used by newest media file grouping feature of Explorer shell with Windows Media Player instead on XP system.

Conclusion: make sure to include shmedia.dll library and register it (it is a COM object so you want to use the regsvr32 tool at run time or simply add a FBA DLL/COM Registration resource command in TD.
Note that the shmedia.dll library is surely in the XPe repository because of the previous versions of the player component.

Wednesday, May 23, 2007

XPe tip #41: "Processor scheduling" registry setting

How to preset Processor scheduling settings in TD?
These settings you can get to from Control Panel-->System-->Advanced-->Settings (Performance)-->Advanced (Processor scheduling section).
Answered here.

The Processor scheduling settings are mapped to the following registry value:
[HKLM\SYSTEM\CurrentControlSet\Control\PriorityControl],"Win32PrioritySeparation".

You can overwrite this value in the Registry Data section of the "Client / Server Runtime (Console)" component.

This binary mask value should be set to (or just include) 0x18 to optimize the system to use a greater share of processor time to run background services.

The mask can actually be tuned much more precisely with setting particular bits of the value. More info on the bits meanings can be found here.
For instance. Clicking Background services sets the value of this entry to 011000 (0x18), and provides for longer, fixed-length processor intervals in which foreground processes and background processes get equal processor priority.

XPe tip #40: "Adjust for best performance" registry setting

How to preset Visual Effects settings in TD? These settings you can get to from Control Panel-->System-->Advanced-->Settings (Performance)-->Visual Effects.
Answered here.

The Visual Effects settings are mapped to the following registry value:
[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\VisualEffects],"VisualFXSetting".
That flag should be set to 2 to optimize the system for best performance. Value 1 is for best appearance.

The same value could be set under HKEY_LOCAL_MACHINE branch to affect all user accounts (global policy). Or under HKEY_USERS\.DEFAULT branch to effect the default user accounts as well as all other user accounts since on XPe they all use Default User settings for defaults during FBA.

No component in TD brings in that registry value by default. It is probably getting [re]set during FBA.

XPe tip #39: Prevent BSOD from locking device

There was a topic we have covered in the newsgroup long time ago I thought I'd want to have listed here as a tip.

We are all familiar with XP famous BSOD - blue screen of death - a stop error displayed after encountering a critical system error. BSOD on 2K/XP is basically first chance exception within the kernel space (the kernel code or a kernel driver) that the system cannot recover and therefore has to shutdown the system. Instead they first show the exception info and, if set up properly, dumping the memory to a crash dump file.

How to prevent the system from showing BSOD? That may be important on an embedded device that is setup to be headless or does not require user interaction during normal hours of operation.

Only way to do that I know of is by setting the following registry entry:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl],"AutoReboot"=dword:1
that is mapped to XP's Control Panel/System/Advanced/Settings (Startup & Recovery)/System Failure/Check-Automatically Restart GUI element.

This registry value belongs to "Computer Prototype" component and in TD can be set (modified) under HAL (PC) component registry data section (e.g., "Standard PC" or "Advanced Configuration and Power Interface (ACPI)").

Sunday, May 20, 2007

XPe tip #38: Ramdisk drivers

Newbies on XPe often confuse the "Windows Ram Disk driver" component with a ramdisk that they may want to use a a temporarily file storage for things like Internet Explorer cache and etc. That component, however, is only a ramdisk driver that is used in RAM boot (Remote boot) scenarios and gets loaded by the loader to emulate the system running off a ramdisk (the entire image is copied to RAM and launched from there).
More info about the RAM boot can be found here.

There are quite many ramdisk driver for creating a device that can be used as a temporary storage available on the Web and created for Windows 2K/XP. For instance, the one from picobay.com.

However, I prefer not dealing with extra software licensing on XPe and tend to use sample driver from sources like Microsoft, CodeProject.com, CodeGuru.com and such where I can always modify the driver to meet my system requirements or simply compile it and use for free.
Here is a couple driver with their sources I used on XPe:
- Microsoft RamDisk sample driver.
Note: this one was created for Windows 2000 but an extremely easy to port to XP and the require porting steps are mentioned in the knowledge base article linked.

- CodeGuru's driver.

Note that for both drivers there are some settings (the ramdisk size and so on) that you may want to preset in the registry while integrating into your XPe.

Honestly I have happened to use other 3rd party ramdisk drivers on XPe but those were tied to customer's SKU's, required an additional license costs and not to be listed here. More ramdisk driver sources I mentioned in the newsgroup.

XPe tip #37: mpnotify.exe is not held by any component

There was an important and interesting post in the newsgroup recently that I have missed due to the MEDC and WinHEC conference attending.

The problem posted was about some API function (such as NPLogonNotify) not working under a full blown XPProEmulation XPe image. Obviously the issue was there not due to a missing component since XPProEmulation virtually includes all software components from the XPe database.
From the newsgroup thread you will see that the debugging the poster team has done showed that their own credential manager was working fine on Desktop OS and only failed to get notifications under XPe with interactive logon.

It appeared that the issue was there due to missing mpnotify.exe - the executable that is responsible for issuing Winlogon notification to registered network credential providers. The NPLogonNotify won't work if mpnotify.exe is not running (e.g., missing or crashed). Basically, it is the mpnotify.exe process who calls the NPLogonNotify function of the network provider .dll file.
The above fact is perfectly documented in this Microsoft Knowledge Base article.

The problems with XP Embedded is that although the mpnotify.exeis in the XPe Repository it doesn't belong to any component. It is basically left out from the component database which makes it harder to include in the image if you don't know what is required to get Network Providers to work and being notified by Winlogon.

A couple more useful links for the proper credential manager Dll integration into your image:
- Required authentication registry keys
- A very good guide written for Logon Filters - a sample credential manager from CodeProject.com.

Saturday, May 19, 2007

XPe tip #36: Changing IP with EWF/FBWF On

There was an interesting topic recently covered in the newsgroup where I had to spend some time to answer the poster questions and got some new knowledge (as very usual while answering other's questions in the newsgroup, btw :-) ) about Microsoft's Registry Filter component.

I won't go into details of that conversation (read in on the newsgroup archive if you wish) but I will just summarize the task, the problem and the solution.

Prerequisites: We have an embedded device running XP Embedded image with EWF or FBWF filter protecting the system partition including system registry hives.

The task: For a person administrating the device (IT, end user, whoever with the sufficient rights to access the network stack on the device) it is required to change system network settings to either switch to required static IP or, if already a static IP used, change the IP address.

The problem: With the write filter protecting the system registry it will be required to commit the entire system registry hive in order to make the IP address change persistent. With EWF it is even worse - you have to commit the entire system volume to accomplish that. The bad part there is that along with the IP changed you will commit all user modifications he might have done on the device and in the system registry. Often, those modifications are undesirable and lead to an unstable device (the beauty of the WF protection of the system volume is getting lost). I am not even touching the sensitive area of viruses and malicious software that a networked device might be exposed to.

The solution: What if we only commit the changes related to the IP settings in registry (what those are? see below...) and leave the system with the fully protected system volume yet.
Yes, it is possible with the component called Registry Filter which has become public within Feature Pack 2007 (there was a QFE for SP2 release that brought in the Registry Filter but that QFE was only available to OEMs).

The Registry Filter was only created by Microsoft to fix the famous issues with Domain Secret Key and TSCAL (license for Terminal Server client). However, the way the component works allows you (undocumentaly, of course) to use it in the purpose of working around some other issues with Write Filter protecting the system volume.

The way Registry Filter works is that it allows you to specify registry keys to make them persistent across reboots while the write filter is ON. Basically Registry Filter creates and initializes a file-backed ramdisk - the hidden system regfData file under the root of the system volume. On that ramdisk it stores files that contain the selected key contents.

My suspicion (as I indicated here) is that the way the \regfData gets away from the EWF protection is through a mechanism similar to the EwfMgrCommitFile. On FBWF, of course, it is much easier to by-pass the protection for the selected file since this filter is file system aware and doesn't require a hack like the EwfMgrCommitFile API.

Now, when we know how the Registry Filter works, lets try to use its functionality to "unprotect" other registry keys we may need to may persistent across reboots with the write filter enabled on the system volume and system registry hives.
Sample registry entries that have to be added to your XPe image at the design time (at run time of design) I already showed in the newgroup. (Note: now we know the time zone info is not the right key for monitoring because of the way Registry Filter gets started - it gets started after the system checks for time)
Here is a sample for reference:
[HKEY_LOCAL_MACHINE\ControlSet001\Services\RegFilter\Parameters\MonitoredKe­ys\2]
"ClassKey"="HKLM"
"FileNameForSaving"="YourFileName.rgf"
"RelativeKeyName"="Software\YourRegistryBranch"

But what are the keys that have to be set for monitoring by Registry Filter to make the IP change persistent?
Initially I thought these:
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip\Parameters\Interfaces\{}],"IPAddress"
and
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\{}\Parameters\Tcpip],"IPAddress"

But after I've done some tcpip driver stack debugging it is clear to me that the first key is the important one. The second will be auto-reset by the system (tcpip driver will refresh the values there grabbing the values from the first key).

So, cool, this means that we just have to add in registry at run time something like the following and it should work:
[HKEY_LOCAL_MACHINE\ControlSet001\Services\RegFilter\Parameters\MonitoredKe­ys\2]
"ClassKey"="HKLM"
"FileNameForSaving"="TcpIp.rgf"

"RelativeKeyName"="SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\{}"

Note1: The CurrentControlSet is used here correctly. But in case you like the actual ControlSet00X key please make sure you work with the right one that gets used for the hardware configuration you are working with (typically the ControlSet001 on the XPe images by default).
Note2: Replace the interface GUID with the real GUID you can read off the registry at run time and that gets generated unique for your NIC adapter and interface.

The problem #2: No, the above didn't work and that is what the initial poster indicated.
The IP gets changed and the system is ping'able. However, as soon as you reboot the ipconfig.exe tool will report the old IP address that you had before the IP change. While the Tcpip GUI (the UI you get to when you go to the network connection properties) or netsh commands report the new IP. I should also note here that the actual IP of the system that you can feel on the network is still the old IP.
Why?

The upgraded solution: After some debugging I've done for the tcpip stack on XP as well as the Registry Filter and EWF components the issue appeared to be related to the loading order of the filter driver. Let me quote myself from the newsgroup.

First I created an image where I could perfectly repro the problem (Minlogon, netsh, EWF, RegFilter). Then I debugged some system stacks to understand what's going on there when we change the IP with the RegFilter on and set to monitor the keys I mentioned earlier. (I picked up EWF vs FBWF to simplify the debugging, having FBWF there won't change the picture)

I must admit the result was a bit surprising to me. Despite the fact that the behavior of the protected system with IP changed and rebooted was quite strange - no sync in the results from ipconfig.exe tool and "netsh interface ip show address" command (the same as you'd do with TCPIP GUI) - the source of the problem was not in the Tcpip stack or the registry keys picked up for the monitoring. The tcpip driver (or a driver from the ipv4 stack) indeed reads the network adapters info into a buffer (link list) at initialization from the registry's Tcpip\Interfaces key. This, of course, includes the IP address settings. When you do "ipconfig /all" it doesn't really go into the registry again but rather queries the driver for that info stored in the cache (buffer). While the netsh commands are more "dynamic" and will scan and return the results from the registry.

It is important to understand that RegFilter doesn't make the monitored keys transparent to the registry hives. The RegFilter rather applies the new registry values - the values that were saved in the monitored key hives on the protected system volume's ramdisk - to the image's registry hives at the next boot time. Some time during the driver init it loads the ramdisk and reads the files on it. Then it sets the actual values in registry to whatever values it reads from the file contents on ramdisk.

The tcpip driver will populate the internal stack buffer very early during the system boot. Looks like the initialization actually happens before the RegFilter is loaded. This leads to the Tcpip structures populated with the old data but the registry info already reflected the new data (laid out by the RegFilter) after the boot. So we end up with the IP addresses reported out of sync between the ipconfig and the registry.

To summarize the above, the real problem there seems to be that the Regfilter and one of the drivers from Tcpip stack belong to the same driver load group. Without the RegFilter enabled the IP would be changed properly (assuming the change can be flushed). This fact is extremely easy to prove with the following simple experiment.
Forgot for a minute about the RegFilter. In fact remove the Tcpip related registry paths from its MonitorKeys branch, commit EWF and gracefully shutdown the image (now the RegFilter doesn't "unprotect" the tcpip keys). Load up the image SYSTEM hive offline (open it with a regedit on a XP Pro machine). Go to the [HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip\Parameters\Interfaces\{}] key and modify the value of the "IPAddress" property there. Unload the hive and boot to the embedded image and verify that the address has changed and is in sync between ipconfig and netsh results (or GUI). Note that you don't even have to mess with the [HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\{}\Parameters\Tcpip] key as it will be automatically updated during the boot (tcpip driver does that at the init time).

It is also easy to confirm with a hex editor that the ramdisk image from the runtime contains the new registry value (new IP address) but the system registry hive has gotten the old one (and this is ok when you think about the Regfilter architecture and design).

So, what's left to fix the solution we created is to change the loading order of the interfering drivers. We can't mess much around the RegFilter driver - it is very dependent on the EWF/FBWF and, the same time, must be launched early at the system boot. So we only can fix the issue by moving the tcpip driver to load it later than the RegFilter and thus pick up the right values from registry hives.

The easiest way, perhaps, to accomplish that would be starting the tcpip driver by the Service Control Manager (SCM) instead of the kernel. The following value would do that:
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip],"Start"=dword:2.
That will put the driver in the auto-start group (started by the service control manager) instead of the system started (kernel or loader started).
This is a pretty safe change unless you've got some custom drivers in your image that demand the network connectivity (IP) very early at the system boot.

With the changes above committed (if you mess with all this at run time), you can try to change the IP and reboot (without an extra commit) and the IP should be the correct one reported by the netsh (or GUI) and the ipconfig as well. At least this is what I am seeing working here.

Just a side note: Domain Secret Key and TSCAL issues were "easier" to fix for Microsoft with the RegFilter approach since those keys are only read late at the boot process (by Winlogon and Terminal Services components).

Thursday, May 10, 2007

MCE tip #3: Filter "More Programs" items

I had a need to hide some items under "More Programs" menu in Vista Media Center app.
Unfortunately there appeared to be no way to create a real filter that would help me to apply some "programmatic" criteria to such area of MCE. However, I just found the simplest way to hide items there. All you need to do is to remove the entry points (application registered in MCE) from the following key:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Extensibility\Entry Points]

For instance, I knews that the following registry key:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Extensibility\Entry Points\{115EADF1-41C4-471b-8FE5-7A52B91BFE75}]
is the entrypoint for Chess Game item of the "More Programs" menu. It is easy to prove if you look under the key and find the "Run" value there. It points to %ProgramFiles%\Microsoft Games\Chess\ChessMCE.lnk shortcut which is obviously responsible for launching the Chess Game.
All I had to do to hide the Chess is to remove the entrypoint registry key.

The main reason I wanted to hide the game items (or any other registered program items) is that some of them still don't comply to the 10 foot UI requirements. Some games are totally 2' UI based and require mouse and keyboard - not sure who decided to put those in the MCE which is supposed to target TV users as well (and hopefully soon will not target PC users!).

Wednesday, May 9, 2007

MCE tip #2: How to launch Media Center app directly to the setup screen

There a re a few places on the web (e.g., The Green Button, Media Center Sandbox Blog, driverheaven.net, etc.) that discussed various way how to launch Media Center and direct it to particular applications. Two most command ways - command line arguments (directmedia and homepage switches) or sending a Win32 message to the MCE window with certain wParam and lParam settings.

I have read all the info and played with it myself. Too many limitations there. Basically you can only redirect the MCE to some [very few] applications - to Live TV, Video Library, etc..

A hacky way I tried - to grab the XML resource names off the MCE binaries - didn't work well. Most of the XML pages there require some machine state variables being passed from the main application entrypoint and this makes the MCE unstable. Mostly it just crashes at the launch.

Only helpful XML page I found I could bring up that way, that wasn't mentioned on the Web, was the Media Center Setup page. All you have to do is to launch MCE with the following command line:
%SystemRoot%\ehome\ehshell.exe /homepage:Options.SetupMediaCenter.xml /PushStartPage:True

Extending and fixing MCE

What you can do if you want to change some functional behaviour of MCE (Media Center application) or add some more functionality there? Or what you can do if you need to fix some 2 or 10 foot UI issues of the Media Center app?

Well, the only way I could come up with is - a global hook or Dll-injection. There are a few techniques known in Win32 world (I wish those exist in Windows CE as well :-) ) how to do the Dll injections (from writing to another process address space to the supported global hooks). Despite the fact that it may be a dangerous implementation (stability issues, lowering security, etc.) this is the only way that seems viable. Two main reasons:
- to have a better and faster implementation for your own monitoring agent application you'd probably want to stick to C/C++ to easier the access to Win32 APIs (P-Invoke's are costly and require lots of coding for missing header). MCE SDK doesn't allow you to write add-ins in native C++. (or perhaps it does but I didn't investigate this deep enough)
- too many limitations there on background add-ins for Vista MCE. You will always lose the MCE launch moment and some other events as well.

I have implemented a MCE hook-up agent that keeps monitoring the Media Center app on a complex of things - messages, windows, positions, etc. - including WM_INPUT (raw input messages), keyboard and mouse messages and etc.

A piece of advice - if you are playing with MCE on Vista, try to stay away from a service implementation for your app. On Vista, services are running in a separate, isolated Windows session (session 0) and it may take you an effort to implement the cross-session interface between your hook into MCE and the service. (not impossible but just over-complicates the implementation)

Tuesday, May 8, 2007

MVP Heroes Campaign

My photo got published on MSDN again :-)
I got a chance to participate in MVP Heroes Campaign program. My take-out from the campaign - Microsoft has an awesome Studio!

Monday, May 7, 2007

XPe tip #35: Language Packs

Recently I have forwarded a reply from Microsoft to the newsgroup in response to the question whether XPe supports Ukrainian MUI. The reply was negative - no, XPe doesn't have the Ukrainian language pack despite of the fact that there is XP Pro Language Pack exists for Ukraine locale.

Basically this means not all Language Packs that have been created for XP Professional were "ported" (componentized) for XP Embedded. Complete list of all the Language Packs for XPe (RTM, SP1, SP2) can be found here.

Tuesday, April 24, 2007

FP2007: What is it?

I liked that straightforward question posted in the newsgroup.

In fact, FP2007 - Feature Pack 2007 - is a nice addition to the SP2 release of XP Embedded tools. It does have some new features that were never rolled out as QFEs for SP1.

More info on the Pack can be found on the product team blog.
The entire XPe documentation tree has been moved to FP2007.

FP2007 is free for XPe SP2 Toolkit owners.
Non-eval version is out there on Extranet website of Microsoft (ECE). Used to be available on OEM Secure Website that was unfortunately deprecated by Microsoft.

The best thing I loved about FP2007 is the stability of the toolkit. I can finally enjoy working with XPProEmulation and other huge projects and the development tools are not crashing. I have checked - it only required to change a few lines of VB script code that runs within TD (Target Designer) to fix the VB memory related issues while loading a huge number of components.
However, there are still some bugs in FP2007 about the toolkit. For instance, it doesn't like setting big pagefile on some ACPI HAL components. Please refer to this thread for more info posted on that bug.

XPe tip #34: Domain Security Key

What is the Domain Secret Key that many times was mentioned in the newsgroup and even required a special fix for EWF to make persistent the key in domain environment? Going forward quickly want to mention the EWF fix - Registry Filter that allow to make some registry keys persistent with EWF enabled and protecting system volume including registry hives.

I am actually not sure who came up with the name "Domain Secret Key". Perhaps, XPe product guys :-) Domain Secret Key is in fact nothing more but the machine's computer account password in a domain. Basically machines - PCs in the network - can be made members of groups just like users can. They log in to the domain just as users do by proving knowledge of their master key, which each machine that is a member of a domain stores in its registry.

So where does Windows store that key? That is very easy to find out (Web Search or the Registry Filter component's script) - [HKLM\Security\Policy\Secrets\$MACHINE.ACC]. If you want to explore that key, run the registry editor as LocalSystem account (the easiest way to do this is from an administrative command prompt; type "at time /interactive regedit.exe", substituting whatever time is appropriate: make it one minute in the future, or use atnow.exe tool I posted the info about in an earlier blog post). Once regedit fires up, carefully look at the subkeys under [HKLM\Security\Policy\Secrets]. In this very key Windows stores a lot of secrets that need to be kept in plaint text for further usage, including service account passwords. You're looking at the machine's password cache, aka LSA private data store, and if the machine is a domain participant, you should see a subkey called $MACHINE.ACC, where the machine's master key lives in.

Why Registry Filter was requested? Specifically to fix the Domain Secret Key issue (and one more key issue irrelevant to this post). In domain environment the default policy is to change the computer account password every 30 days. If EWF is enabled, after 30 days you may end up with the device not being able to logon to the same domain it has joined to. So the Domain Secret Key needs to be made persistent in registry to avoid password age issues in domain environment.

EWF Registry Filter creates and initializes a file-backed ramdisk used. On that ramdisk it stores files that contain the selected key contents. The secret policy protection key file is named MacAcc.rgf.

Below is just my idea on how things could be fixed if no Registry Filter was acceptable.

Using device's Local Security Policies you can change the password age:
[HKLM\SYSTEM\CurrentControlSet\Services\NetLogon\Parameters],"MaximumPasswordAge"=dword: (default 30 days)

You can also disable the password change completely:
[HKLM\SYSTEM\CurrentControlSet\Services\NetLogon\Parameters],"DisablePasswordChange"=dword:1
Or refuse to domain controller's change of the computer password:
[HKLM\SYSTEM\CurrentControlSet\Services\NetLogon\Parameters],"RefusePasswordChange"=dword:1
Corresponding local security policies are:
Domain Member: Maximum machine account password age (MaximumPasswordAge)
Domain Member: Disable machine account password changes (DisablePasswordChange)
Domain Controller: Refuse machine account password changes (RefusePasswordChange)

You can see them if you launch gpedit.msc (Group Policy Editor) and go to Local Computer Policy-->Computer Configuration-->Windows Settings-->Security Settings-->Local Policies-->Security Options.

I think in domain environment those policies are getting pushed to clients from domain controller. So a domain admin can always change those on the server side before turning on the policy on multiple clients.

Some more info you can find in the following MS KB articles: KB175468 - Effects of machine account replication on a domain, KB154501 - How to disable automatic machine account password changes, TechNet article about MaximumPasswordAge.

Sunday, April 22, 2007

XPe tip #33: New hardware UI

What component is responsible for the UI you see when a new hardware is attached and detected on your target PC?
The answer is simple - newdev.dll - Add Hardware Device Library. The newdev.dll library can be brought in your image if you add "Add Hardware Control Panel" component.

That library implements all the GUI for the Add New Hardware wizard. It is also getting called from Device Manager when you are trying to Update a driver. It is interesting though the Device Manager does not seem to explicitly depend on the Add Hardware Control Panel component or, at least, newdev.dll (as a rawdep). I'd call this a bug.

If the newdev.dll removed, it gives you an image that is deaf to installing new hardware. Very useful feature for headless devices.

Saturday, April 21, 2007

XPe tip #32: Best practices for power interrupt safety on headless devices

What if you are creating a headless device (truly headless - with no VGA - or just with no console devices attached such as keyboard and mouse) and you want to make sure the OS runs stable and reliable if an power interrupt occurs.

Below is list of things you may want to look at in the XPe documentation or that make common sense.

- Use UPS (Uninterpretable Power Source).

- Make use of XPe Write Filters: EWF/FBWF - protecting the storage media with write filters will make it more reliable for the system to recover: http://msdn2.microsoft.com/en-us/library/ms912906.aspx, http://msdn2.microsoft.com/en-us/library/aa940926.aspx.

- Headless support - this is what you may need if no console devices are attached to or supported by your hardware: http://msdn2.microsoft.com/en-us/library/ms932872.aspx.
Also, it may be a good idea to remove newdev.dll library (Add Hardware Control Panel component) from your image so that there is no GUI fired up if new hardware is attached to the target device.

- Don't include Safe Mode and Dr. Watson components in your image.(http://msdn2.microsoft.com/en-us/library/ms940820.aspx)

- Setup the system crash dump procedure to restart the PC automatically when a GPF occurs. Set the following reg.entry:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl],"AutoReboot"=dword:1

- Get rid of all [most of] system popups and message boxes that may get your system GUI unintentionally locked:
http://msdn2.microsoft.com/en-us/embedded/aa731206.aspx,
http://msdn2.microsoft.com/en-us/library/ms933122.aspx.

- It often makes sense to create your own custom shell for the target XPe OS where you can control the GUI behavior (to make it more "headless compliant") and get rid of Explorer Shell that is not really headless friendly: http://msdn2.microsoft.com/en-us/library/ms913692.aspx.

Friday, April 20, 2007

XPe tip #31: Wallpaper change

Want to change the wallpaper of your Desktop? Just follow this procedure:

The registry entry you want to preset in TD is:
[HKEY_CURRENT_USER\Control Panel\Desktop],"Wallpaper"=Reg_sz:"path to image"

To affect all users:
[HKEY_USERS\.DEFAULT\Control Panel\Desktop],"Wallpaper"=Reg_sz:"path to image"
Note: This entry will also be used as wallpaper in Winlogon time.

That registry value can be overwritten in the registry section of "Win32 API - User" component.

Thursday, April 19, 2007

MEDC 2007 is coming..

MEDC 2007 is coming! I am anxious to go there this year and hear the latest on XPe and CE front. So much info and so little time though.
Sessions info is looking very encouraging and interesting.

AutoPlay without Explorer

I am filing this not as a tip but rather as an encouraging info. Unfortunately, I cannot reveal any technical details of my implementations here at this stage of the project (done for a customer) but more to show up soon, I hope.

It has been a problem for quite some time to get some of the Explorer-only features working on XPe without Explorer shell. Myself I had a negative post to the newsgroup on the subject.
The real challenge appears there not when you remove the Explorer.exe from the image but when you stop launching Explorer as shell.

Removing Explorer gives you a lot:
- Explorer is in general a big security hole.
- It is hard to get rid of all unwanted pieces of the Explorer.
- Explorer is not for embedded (it is not "embedded" type of application and have never been really componentized).
- Explorer increases your image footprint. Explorer dependencies are awfully big.
- Explorer implements only 2 foot UI and it may be very inconsistent to your own applications GUI.
- so on.

Anyway, I finally managed to develop my own shell that implements missing pieces that were only working with Explorer Shell so far. Someday I will post what are the missing features among which is the AutoPlay. But the important message now is that AutoPlay can be working without Explorer shell running in background. You just have to do some hacking and coding within your own shell app.

Another good news is that I got the same shell app up and running on Vista (with some mods though) where I stripped out Explorer components.

XPe tip #30: Where is intl.cpl?


Surprisingly enough but this question keeps coming up in the newsgroup.

The answer is simple: beside the "Regional and Language Options" component that you might have already added to your configuration, please also add "International Control Panel" component that is located under "Software : System : User Interface : Shells : Windows Shell" category.

XPe tip #29: How to prepare and add Group Policy at build time

We all know how Group Policies are helpful in XPe world. It is not a problem to edit the policies at run time with GPEdit (Group Policy Editor) or directly in registry. But who want to repeat the same procedure many times while deploying images during development time?! So, this needs to be somehow automated.
Also, not to mention that the GPEdit with all the dependencies it requires is quite heavy.

But how to set up the policies you want in TD at image design time?
A few way to accomplish that:

- Mess with the Group Policy INF files and get them into your image updated. This is typically more complicated way because it is often not clear how policies described in the INF. Also, you'd need to bring GP updater tool as well.

- Set up most of the policies you need as registry entries and add them in TD via your own component or TD Extra Registry section.
Here you will find a helpful page (Group Policy Settings Reference) where you can download Excel document with most of the Group Policy listed with corresponding registry entries mentioned.

- My favorite. I have provided this answer in newsgroup a while ago.
Set up policy settings on XP Pro machine or on your own working XPe image that includes GPEdit tool and then save the following folder with all its content:
%systemroot%\system32\GroupPolicy.
This folder contents MACHINE and User specific policies set up. Those are just .POL files. It also contains the shutdown/logon/logoff scripts (only if you didn't change the default location for those in GPEdit).

XPe tip #28: Boot.ini Extra Switches


How do you change boot.ini for an XPe image during build time? The answer is - manually :-(

TD does generate boo.ini for you with the default ArcPath based on what you set up on the Configuration page. It is done by the EndOfBuild event handler of the Base Component that basically contains the Configuration build logic. However, if you plan to play with dual boot or want to change a logo or add some other boot.ini switches you have to modify just built boot.ini manually every time you do a build in TD. This is typically not a big deal but annoying, especially if you forgot to change boot.ini just before you deploy the image.


There was a few requests in the newsgroup for a way to change boot.ini content directly in TD. E.g., this one. For some reason Microsoft hesitated to implement that (I suspect the feature got a low priority) so I finally decided to step in and do something for XPe devs in this area.

Since TD doesn't support plug-in interfaces I could only implement that feature through undocumented CMI interfaces. Moreover, I didn't want to rewrite the Base Component (possible and I've done that once for debugging purposes) I decided to add functionality that allows us to add extra boot.ini switches. After all, we could always change ArcPath on the Configuration settings page but never were able to change, say, boot loader logo (the famous bootlogo switch).

I ended up with a script that I added to my XPeTools package that does what I described above. Via an additional field of XPeTools macro component's settings page you can add extra switches to the default ArcPath of boot.ini.

Note: added boot.ini extra switches are persistent and won't be wiped out when you start another session of TD with the same project (or simply close and re-open the project in TD). This is done the same way as I did that for other components of XPeTools package. They all have some advanced component properties used as a storage for the component functionality required settings.




A screenshot of TD with the boot.ini extra switches page open is attached.

Saturday, April 14, 2007

XPeTools component update

Thanks to Jim, a documentation bug was discovered in the DependencyExplorer component.
I now had some time to go over that bug and fix it in the new version of the component.
I uploaded the updated package of XPeTools on www.xpefiles.com. Updated package also includes some bug fixing for XPProEmulation component in UI and script areas. From UI I have removed the leftover Active button, assigned name to the Group filters to remove the confusion caused by the undocumented exclusion filter, fixed other minor UI issues.

Friday, April 13, 2007

XPe tip #27: PID - how to use?

This is an old topic but unbelivably comes online once in a while.

XPe devs should understand the difference between runtime licenses and licensed XPe Toolkit.
Runtime license is provided to you after you signed a legal agreement with Microsoft (through XPe distributor) called CLA. Then you are given stickers with PID numbers on it (numbers like xxxxx-xxxxx-xxxxx-xxxxx-xxxxx). Used to be that the stickers all had different unique PIDs, nowadays they are the same in one multiple-license volume purchase. Depending on how many licenses are covered by your agreement you should be getting some amount of stickers. You use just one PID from any sticker and type it in the Runtime ID field on Configuration Settings page in TD. This will get rid of the "Evaluation build ..." text at run time (it other words, your runtime will be properly activated).

Another story with the toolkit. To be able to create activated runtimes, to even see that Runtime ID field in TD, you must be having and using the licensed toolkit. In Eval toolkit (the one you can download from public MSDN or receive on Eval CDs) that field is hidden, thus not allowing you to create activated runtimes. The CD-key you are getting from Microsoft (aka the toolkit Serial Number) is to be used once when you are installing the tools (TD, CD, database, etc.). Don't put this number in the Runtime field in TD - it is a wrong place for it. Rather type in there a PID from a sticker.

Also, just FYI. In SP2 times there was a QFE issued by Microsoft that was patching the toolkit to be able to use new PID range in SP1 database. That QFE was to address Extend Runtime Product Key Range for Windows XP Embedded. Quote: "This update allows OEM's and embedded developers to use runtime product keys provided after the release Windows XP Embedded with Service Pack 2 on a database containing Windows XP Embedded Service Pack 1 or earlier component databases.".

If you contact your XPe distributor, they should be able to explain you all the details about PId and etc.

XPe boot-wish-list

To continue my post about MBR problems.

I believe XPe documentation doesn't describe this topic in the details it should have to. In some places in the documentation they assume you've got XP Pro running on the device. Which means you've already properly partitioned and formatted the local storage to boot from using XP Pro Setup.
Below is my personal wish-list about what should be included in the next Service Pack for XPe if Microsoft will release one:
- Create one tool (better Win32 app that we can launch from WinPE, XP Pro or any other flavor of XP and nowadays Vista) that will properly partition local storage on target device. After all, XP Pro Setup does that *much* better and more reliable then any tools from XPE toolkit.

- Provide more detail troubleshooting section in the documentation that will cover as many failed case as possible - from broken MBR to boot sector, from changing timeout of boot.ini to debug parameters for KD.

- There should be documentation section (part of XPe docs!!) that explain how BIOS loads the MBR, info about MBR structure, boot sector, boot sequence of XP, loader (how it gets loaded by bootsector) and etc. Ideally they should include info about all the fields of MS version of MBR and boot sectors. After all, most of this info is out there in MS KB articles (but hard to search) and on the Web.

- Provide a set of tools (my best wish is to have one complex command line tool with a bunch of switches like myu favorites MbrFix and NirCmd) that will do fixing for all the parts that may get broken - from MBR to bootsectors (all types!) and partition setup (I secretly don't get why we should be using 3rd party tools like MbrFix, for instance?).

I will probably update this list once in a while when my memory flashes out with more ideas I've been having.

XPe tip #26: Msvcr80 in XPe world

The newest Visual Studio (VS2005 and next version Orcas) by default will link all C/C++ applications you create with the IDE wizard to the latest CRT library which is msvcr80.dll. By default, again, it uses a shared library link (dll must be present in the system path on the target).

Only XPe FP2007 brings in the library but as a part of the huge .Net 2.0 Framework component. It is not even wrapped into a separate primitive component. Obviously, you don't want to add such beast as .Net 2.0 to just add msvcr80, unless you need .Net 2.0 in your image.

Here is a few tricks you can use to get rid of possible problems with missing msvcr80 at runtime:
- Copy msvcr80.dll along with msvcm80.dll and msvcp80.dll libraries from VS2005 folders on PC where you run the VS tools (typically somewhere under "\Program Files\Microsoft Visual Studio 8\VC\redist\x86\Microsoft.VC80.CRT").

- Make sure you are working with the proper version of the libraries corresponding to the project settings - Debug or Release. Often, people are working with Debug build (which is default by the wizard, btw) that requires debug version of the msvcr80 - msvcr80d.dll.

- Please copy manifest built along with your application exe in VS2005. This will make sure it will use the right dependencies versions.

- Often (depends on your application logic and implementation) you can simply use only basic Win32 API and you may not need the CRT libraries at all. The MSVCR80 may get linked however if you have "Multi-Threaded DLL (/MD)" or "Multi-Threaded Debug DLL (/MDd)" option set for the Runtime Library on the Code Generation tab in the C/C++ branch of the Project Properties in VS2005. Just use the single-threaded Dlls there and disable Exceptions. Or, alternatively and my favorite, just link statically to the library to remove the dependency at all (use "Multi-Threaded (/MT)" option for instance).

- Make sure to include side-by-side components in TD.

- If the above doesn't help, install VS2005 (x86) Redist package at runtime.

- If the application still fails to load, run the application through DependencyWalker on the target devices. More precisely, run it through Profiler of the DependencyWalker. Then you should be able to see why it's failing to load.

XPe tip #25: Dial-up autoconnect

Adam has recently posted a link to a nice dial-up tip in his blog that I commented a little bit. Thought I would mention the fix in my blog as well but all credits should go to Adam.

To get rid of the annoying Dial-up connection dialog appearing every time an application is calling to create an Internet connection through WinInet API (the exact component responsible is rasdial code) all you have to do is to set the follwoing key below:
[HKEY_CURRENT_USER\RemoteAccess\Profile\<ConnectionName>],"AutoConnect" =dword:0.
Where the ConnectionName is the name you assigned to the dial-up connection created.

This and more info you can read about dial-up autoconnect in this MS KB article.

I worked a lot on the rasdial components in WindowsCE a while ago but on XP those are a bit different structured and, obviously, without the source code you are provided with less control on XP over the RAS than on WinCE.

XPe tip #24: CSDVersion

On versions of XPe toolkit (database) prior to FP2007 update there was a problem related to version info that gets set in the system registry hive and gets returned by APIs like GetVersion[Ex]. Some applications code or application installation scripts make queries for the system version info to either visualize it to end user, change install path or setup or change the application logic if not quit automatically on unsupported OSes.
Obviously, if the version info is set incorrect those application will fail or won't work as expected (as tested on regular XP Pro with the same SP level installed).

The system version is stored in the CSDVersion string registry value under [HKLM\SYSTEM\CurrentControlSet\Control\Windows] key. You can easy find out what the permitted values are but instead just simply match whatever you've got on the same level SP XP Pro machine under the same key.
Some of the right values I mentioned in this thread.

XPe tip #23: Account password expiration

I hate the account password expiration feature on XP. But on XPe image it is even worth because your users may get stuck there forever if their account is expired. Giving the fact that your locked down image may not allow to fix the problem with GUI, the person who is administrating the target devices will hate you even more.

So, how to get rid of the account password expiration? Actually, easy. A few way you can accomplish that depending on what's included in your image already:

- Use GUI User Manager at run time to change that flag.

- Run the following command at run time:
net accounts /maxpwage:unlimited
You can automate and launch it during FBA with FBA Generic command, for instance.

- My favorite: use command line version of User Manager - usrmgr.exe tool from 2K or XP Resource Kit. It supports a command line argument to change the password expiration on selected accounts. E.g.:
cusrmgr.exe -u UserName +s PasswordNeverExpires.

-Use Group Policy (GPO) template to reset the security settings of password expiration timeouts. Here is a nice MS KB article explaining how to do that. The option you are looking for is MaximumPasswordAge under [System Access] section. So, you create a similar INF file, where you set the MaximumPasswordAge to -1 - unlimited, and install it during FBA using secedit.exe tool (Security Configuration Engine Command Line Utility component). Here is a simple INF you could create to get rid of the annoying password expiration feature:
[Version]
signature="$CHICAGO$"
[System Access]
MaximumPasswordAge = -1

XPe tip #22: Disk read errors

Perhaps, most of us who ever tried to partition a drive manually (not using XP Setup CD) have seen a boot time error where it says "disk read error" or something similar coming from BIOS.

There was quite many thread about that in newsgroup. Here is one of the latest.
I am not going to go deep here and explain why it might have happened if you used Microsoft tools like diskpart and etc. Just keep in mind that tools like diskpart are intended to be used by IT technicians who know their stuff and dealt with similar partitioning problems many times. Now you also do :-)

I found it extremely easy to repro the issue if you use VPC (Microsoft Virtial PC 2005-2007). Just create a new image there and partition the drive with diskpart. Even if you set the system partition active, you may end up with no booting image because of the screwed MBR record. This of course tends to be the case from machine to machine. I mean it is not 100% reproducible if you move to another PC.

The fix is actually quite simple. I posted it many times:
- Mark the system partition as active if you haven't done so.
- Try fixing MBR. You can do it manually (just have to get used to MBR binary flags and partition table settings) or you can use MbrFix tool I mentioned in the newsgroup and in this blog. MbrFix tool is lovely and provides lot more functionality.

XPe tip #21: Dependecy Explorer vs Configuration Explorer

There was recently a bug found in the documentation of the Dependency Explorer component that led to some confusions I had to remove in the newsgroup.

So, what's the difference between Dependency Explorer and Configuration Explorer components.
The DependencyExplorer component is aimed to walk through components in the database. While the ConfigurationExplorer component serves the purpose of walking through components in the Configuration.

If you fire up Settings page of the DependencyExplorer component and type in a component name in the search edit box and hit Apply button you will see the following sections listed in order:
"Components:" section lists components that comply to the search criteria. Even if you typed in an exact component name, you will still likely see a few components listed here as there might have been a few revisions of the same component imported in the database and released.

"Component Dependencies:" section lists components that the component selected (clicked on) in the above section depends on.

"Component Chain:" section lists components in the inheritance chain of the component selected in the first section.
Note: the inheritance chain has nothing to do with the dependency chain.
Here you're supposed to see all the base prototypes of your components, if any, up to the very root one and the component itself.

"Custom Properties:" section lists advanced properties of the DependencyExplorer component (not relevant here).


Now about the ConfigurationExplorer Settings page:

"Configuration Components:" section lists components that comply to the search criteria and belong to the config. Here, if you typed in an exact component name, you will likely see one component listed as it is hard to add a few revisions of the same component to the same configuration (but not impossible!).

"Component Dependencies in Configuration:" section lists components that the component selected in the above section depends on. Only those dependencies listed here that are included in the configuration.

"Configuration Components depend on the selected component:" section lists components that depend on the component selected (clicked on) in the first section but only included in the config. This is similar to the result of the DEPTREE command of XPECMD tool released by Microsoft with FP2007 but not exactly the same. The difference is that the DEPTREE walks through the database while the ConfigurationExplorer also filters out components to list only those that belong to the same Config. If you need the DEPTREE functionality there, you just use ConfigurationExplorer component within a huge config like XPProEmulation project.

"Component Chain:" section lists components in the inheritance chain of the component selected in the first section. Here again you are supposed to see base prototypes of your components, if any. The prototype components are all listed here, regardless of if they all belong to the Config, and the component itself.

"Custom Properties:" section lists advanced properties of the ConfigurationExplorer component (not relevant here).

Now let me show you the results you actually get from if you explore XPProEmulation component within XPProEmulation project using both tools.

DependencyExplorer:
Search box: XPProEmulation

"Components:" section:
XPProEmulation [Version ...], [Visibility=1000]

"Component Dependencies:" section:
No component dependencies found.

"Component Chain:" section:
XPProEmulation [Version ...], [Visibility=1000]
Default Prototype Component [Version ...], [Visibility=1]


"Custom Properties:" section:
....


ConfigurationExplorer:
Search box: XPProEmulation

"Configuration Components:" section:
XPProEmulation [Version ...], [Visibility=1000]

"Component Dependencies in Configuration:" section:
No component dependencies found.

"Configuration Components depend on the selected component"
XPe Tools [Version ...], [Visibility=1000]

"Component Chain:" section:
XPProEmulation [Version ...], [Visibility=1000]
Default Prototype Component [Version ...], [Visibility=1]


"Custom Properties:" section:
....

Hope this clears things up.

XPe tip #20: XPe Tools package

I keep mentioning XPe Tools package from www.xpefiles.com in newsgroup.
So, what is it? It is a set of tools I created for XPe developers to help exploring components and configurations, upgrading to newer version of SP, fighting with component dependencies and so on and so forth.
Actually, the initial purpose I created the first component in the package (XPProEmulation) was to get familar with undocumented CMI interfaces that XPe toolkit uses. Some day I'll write a few notes and hopefull articles here in the blog about CMI and what can be done using it.

The tools are implemented as components themselves (I thought it would be much more useful to get your hands on the tools by not leaving TD environment). More or less TD plug-ins, although TD lacks the real plug-in interface.

When you import the XPeTools.SLD, you will get a few components added to the database and they will appear under "Software : Test & Development" category in the Component Browser in TD. No repository is provided or required.
The new components you are going to see are XPe Tools (macro), Configuration Explorer, Dependency Explorer, Derive Explorer, XPPro Emulation and Smart Upgrade. Every component from the package has a link to an encrypted CMI script and DHTML page representing that component functionality.

You may include all the components into your Configuration, the image won't grow in size since no files added, by resolving dependencies of the macro component "XPe Tools". Please use XPeTools Package that includes all the components mentioned above to operate (delete/add/etc.) on the package components at once.

You will get more details on what components do and how they are implemented if you look at the Readme.txt file provided within the same zip as the SLD file. Also, all the components have links on their Settings pages ("more info") to show you more information about the current component and functionality.

I will briefly describe here what components are and what they do:

XPeTools - macro component that brings in all other components from the package. Some useful global options and flags added. Boot.INI extra switches.

Configuration Explorer - this component allows you to navigate through Configuration components to see their dependencies and components that depend on them. It also allows you to save configurations in plain text files (useful for further project comparisions).

Dependency Explorer - allows you to navigate through XPe database components to see their dependencies. Variaty of component filter options is available.

Derive Explorer - allows you to navigate through XPe database to see derived components. (got to get this worikng on FP2007)

XPPro Emulation - allows you to add group filtered components from XPe database to your Configuration. Inclusion and Exclusion Group Filters are suported. Due to some bugs of SP2 FP2007 (me reported to Microsoft) the exclusion filter is now required in order to get rid of Hardware components from the final Software-only XPProEmulation project.

Smart Upgrade - allows you to upgrade components in the Configuration preserving their extended properties. Smart Upgrade component will allow XPe developers smoothly "port"their SLX projects to SP2 and later preserving the configuration component settings during the upgrade (the settings of the "User Interface Core" component, for instance). This component should work for SP1-->SPx ports as well as for RTM-->SPx.


One quick comment about one option of the DependencyExplorer and other components - Minimum Visibility. This option is very helpful in the filtering feature of the tools. But I mainly use it to lower the visibility below 100 as the TD doesn't really allow that (well, it does but it requires a reg hack that I will post some day).
And please don't hesitate to lower the option value down to 0. Then you will be able to see even Legacy components left in the database from previous version of the product (and still distributed by Microsoft). you should not bother to add those components to your configuration(s) but it is a great knowledge you can gain if you explore those components as well.

XPe tip #20: Active HTML Desktop

An interesting question recently popped up in the newsgroup about how to get a toolbar working in Active HTML Desktop on XPe.

As you know, on XP if you wanted to set up Desktop to show an HTML page you'd go to -> Properties -> Desktop -> Customize Deskop -> Web page and add a web page (aka component) there with soem setting options assigned. When it was done, your Desktop would show the selected HTML page(s). I always use this feature on my XP machines where I set the IE Home page (typically Google.com) to be the Desktop.

This feature works ok on XPe, of course. However, an issue was reported about the toolbar of the Active Desktop not being popping up.
On XP, that toolbar (a popup that appears on the top of the Desktop when you are hovering mouse cursour over the top of the desktop HTML page) gets disappeared when you have the desktop components locked. As I mentioned, components include the web pages you set for the active desktop. The lock can be set with GUI - just open up Desktop Items dialog (Display -> Properties -> Desktop -> Customize Desktop -> Web) and clear the "Lock desktop items" checkbox at the bottom.

But we are always interested in the registry entries mapped to particular XP features when we have to pre-set those in TD for XPe builds. The HTML Desktop Lock is mapped to
[HKCU\Software\Microsoft\Internet Explorer\Desktop\Components],"GeneralFlags" dword registry value. It is a bit mask and corresponds to a few general options of the page display.

If it is set to "2" (or better say, the second bit set) the desktop items are locked and the toolbar won't be sliding out. You should reset it to 0 if you want the toolbar to appear by mouse hover event.

I am going to update this post with some more information as required to clarify a few things about the ActiveDesktop feature of XP.

First of all, the feature works great on XPProEmulation image. Note that to get it to work there didn't require any custom registry entries to be populated.
A few more tips here:
- Shell32.dll must be in the image. Well, since the feature doens' make sense out of Explorer Shell - Desktop is required - it is almost useless to say the Shell32 is required too.

- Make sure "Primitive: Browseui", "HTML Rendering Engine", "Primitive: Oleaut32" and "Primitive: Shlwapi" components are in your image. Last 3 components are probably in your image already if you resolved Explorer Shell dependencies.

- Make sure the user you are logged in have Admin privileges.

- Check if Desktop.htt file gets generated under <%UserProfile%\Application Data\Microsoft\Internet Explorer> folder.

- Make sure "Active Desktop Mover" object is registered. It will be mentioned under [HKEY_CLASSES_ROOT\CLSID\{72267F6A-A6F9-11D0-BC94-00C04FB67863}] key. Basically this object is what you see as the toolbar when you move mouse cursor over the top area of the page.

- Make sure the Low security settings for My Computer (local) Internet explorer zone are set.

To explain you why I ask you to check the above, I will have to explain you how the ActiveDektop feature works with a Web component setup. When you set it up in the Desktop Items dialog, Explorer Shell automatically generates an html based file - Desktop.htt - under a user profile folder (the one mentioned above). That HTT file is the Web page you see on your Desktop afterwards. It may contain a few components (Web pages) - they are all FRAMEs within the HTT html file (tag IFRAME used). The HTT page also has an object embedded on the page (OBJECT tag used) - Active Desktop Mover - that is basically the toolbar you see when you move the mouse cursor over the top side of the page. All the Web components and the toolbar appearances are controled by a set of values and subkeys under [HKCU\Software\Microsoft\Internet Explorer\Desktop\Components] key.

Please note that you can open that Desktop.htt file within IE as a regular dhtml page - you will notice it looks very like your Desktop. Obviously, there is no default association for .htt file on XP(e) so you will have to either manually associate it with IE or simply launch the following command line:
iexplore "%UserProfile%\Application Data\Microsoft\Internet Explorer\Desktop.htt"
(replace the %UserProfile% with the actual value of the env.var before the launch)

When launched, you will have to allow IE to render the page with the ActiveX control embedded (the Active Desktop Mover control) via a little popup appearing as the IE info bar.

All the above applies to IE 5.5 and any later version.

What launch in IE gives you? A perfect tool to test whether the toolbar appears or not when you move the mouse cursor over the top of the page. You can fire Regmon/Filemon (better Process Monitor) while running IE instance and monitor what's going on there when you navigate the browser to the HTT page. It will show you all the object Dlls loaded, registry access, etc. Or, better, run the above command line and the mouse hover test through DependencyWalker Profiler (open \program files\Internet Explorer\iexplore.exe, hit F7, type in the full path to HTT file and hit OK). It will show you in red if any Dlls is missing while you are testing the page.


Last update on the issue. When you target the feature for your XPe image and don't want to use a full blown image like XPProEmulation, please make sure you include the template for Desktop.htt. The template is going to be used by the shell to generate the actual Desktop.htt with the list of components you place on your Desktop. If you don't include the template in your image, Web Desktop feature implementation code will use a dummy HTT file that won't have a reference to the toolbar ActiveX component and therefore won't show it on mouse hover event.
Easieast way to get the HTT file in your image is to add "Images and Media" component. However, the component is quite big so you may end up just adding the HTT file only via your own component or TD Extra Files section. That file is is the XPe Repository anyway.


Good luck with debugging Web components on your ActriveDesktop!