Tuesday, April 24, 2007

FP2007: What is it?

I liked that straightforward question posted in the newsgroup.

In fact, FP2007 - Feature Pack 2007 - is a nice addition to the SP2 release of XP Embedded tools. It does have some new features that were never rolled out as QFEs for SP1.

More info on the Pack can be found on the product team blog.
The entire XPe documentation tree has been moved to FP2007.

FP2007 is free for XPe SP2 Toolkit owners.
Non-eval version is out there on Extranet website of Microsoft (ECE). Used to be available on OEM Secure Website that was unfortunately deprecated by Microsoft.

The best thing I loved about FP2007 is the stability of the toolkit. I can finally enjoy working with XPProEmulation and other huge projects and the development tools are not crashing. I have checked - it only required to change a few lines of VB script code that runs within TD (Target Designer) to fix the VB memory related issues while loading a huge number of components.
However, there are still some bugs in FP2007 about the toolkit. For instance, it doesn't like setting big pagefile on some ACPI HAL components. Please refer to this thread for more info posted on that bug.

XPe tip #34: Domain Security Key

What is the Domain Secret Key that many times was mentioned in the newsgroup and even required a special fix for EWF to make persistent the key in domain environment? Going forward quickly want to mention the EWF fix - Registry Filter that allow to make some registry keys persistent with EWF enabled and protecting system volume including registry hives.

I am actually not sure who came up with the name "Domain Secret Key". Perhaps, XPe product guys :-) Domain Secret Key is in fact nothing more but the machine's computer account password in a domain. Basically machines - PCs in the network - can be made members of groups just like users can. They log in to the domain just as users do by proving knowledge of their master key, which each machine that is a member of a domain stores in its registry.

So where does Windows store that key? That is very easy to find out (Web Search or the Registry Filter component's script) - [HKLM\Security\Policy\Secrets\$MACHINE.ACC]. If you want to explore that key, run the registry editor as LocalSystem account (the easiest way to do this is from an administrative command prompt; type "at time /interactive regedit.exe", substituting whatever time is appropriate: make it one minute in the future, or use atnow.exe tool I posted the info about in an earlier blog post). Once regedit fires up, carefully look at the subkeys under [HKLM\Security\Policy\Secrets]. In this very key Windows stores a lot of secrets that need to be kept in plaint text for further usage, including service account passwords. You're looking at the machine's password cache, aka LSA private data store, and if the machine is a domain participant, you should see a subkey called $MACHINE.ACC, where the machine's master key lives in.

Why Registry Filter was requested? Specifically to fix the Domain Secret Key issue (and one more key issue irrelevant to this post). In domain environment the default policy is to change the computer account password every 30 days. If EWF is enabled, after 30 days you may end up with the device not being able to logon to the same domain it has joined to. So the Domain Secret Key needs to be made persistent in registry to avoid password age issues in domain environment.

EWF Registry Filter creates and initializes a file-backed ramdisk used. On that ramdisk it stores files that contain the selected key contents. The secret policy protection key file is named MacAcc.rgf.

Below is just my idea on how things could be fixed if no Registry Filter was acceptable.

Using device's Local Security Policies you can change the password age:
[HKLM\SYSTEM\CurrentControlSet\Services\NetLogon\Parameters],"MaximumPasswordAge"=dword: (default 30 days)

You can also disable the password change completely:
[HKLM\SYSTEM\CurrentControlSet\Services\NetLogon\Parameters],"DisablePasswordChange"=dword:1
Or refuse to domain controller's change of the computer password:
[HKLM\SYSTEM\CurrentControlSet\Services\NetLogon\Parameters],"RefusePasswordChange"=dword:1
Corresponding local security policies are:
Domain Member: Maximum machine account password age (MaximumPasswordAge)
Domain Member: Disable machine account password changes (DisablePasswordChange)
Domain Controller: Refuse machine account password changes (RefusePasswordChange)

You can see them if you launch gpedit.msc (Group Policy Editor) and go to Local Computer Policy-->Computer Configuration-->Windows Settings-->Security Settings-->Local Policies-->Security Options.

I think in domain environment those policies are getting pushed to clients from domain controller. So a domain admin can always change those on the server side before turning on the policy on multiple clients.

Some more info you can find in the following MS KB articles: KB175468 - Effects of machine account replication on a domain, KB154501 - How to disable automatic machine account password changes, TechNet article about MaximumPasswordAge.

Sunday, April 22, 2007

XPe tip #33: New hardware UI

What component is responsible for the UI you see when a new hardware is attached and detected on your target PC?
The answer is simple - newdev.dll - Add Hardware Device Library. The newdev.dll library can be brought in your image if you add "Add Hardware Control Panel" component.

That library implements all the GUI for the Add New Hardware wizard. It is also getting called from Device Manager when you are trying to Update a driver. It is interesting though the Device Manager does not seem to explicitly depend on the Add Hardware Control Panel component or, at least, newdev.dll (as a rawdep). I'd call this a bug.

If the newdev.dll removed, it gives you an image that is deaf to installing new hardware. Very useful feature for headless devices.

Saturday, April 21, 2007

XPe tip #32: Best practices for power interrupt safety on headless devices

What if you are creating a headless device (truly headless - with no VGA - or just with no console devices attached such as keyboard and mouse) and you want to make sure the OS runs stable and reliable if an power interrupt occurs.

Below is list of things you may want to look at in the XPe documentation or that make common sense.

- Use UPS (Uninterpretable Power Source).

- Make use of XPe Write Filters: EWF/FBWF - protecting the storage media with write filters will make it more reliable for the system to recover: http://msdn2.microsoft.com/en-us/library/ms912906.aspx, http://msdn2.microsoft.com/en-us/library/aa940926.aspx.

- Headless support - this is what you may need if no console devices are attached to or supported by your hardware: http://msdn2.microsoft.com/en-us/library/ms932872.aspx.
Also, it may be a good idea to remove newdev.dll library (Add Hardware Control Panel component) from your image so that there is no GUI fired up if new hardware is attached to the target device.

- Don't include Safe Mode and Dr. Watson components in your image.(http://msdn2.microsoft.com/en-us/library/ms940820.aspx)

- Setup the system crash dump procedure to restart the PC automatically when a GPF occurs. Set the following reg.entry:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\CrashControl],"AutoReboot"=dword:1

- Get rid of all [most of] system popups and message boxes that may get your system GUI unintentionally locked:
http://msdn2.microsoft.com/en-us/embedded/aa731206.aspx,
http://msdn2.microsoft.com/en-us/library/ms933122.aspx.

- It often makes sense to create your own custom shell for the target XPe OS where you can control the GUI behavior (to make it more "headless compliant") and get rid of Explorer Shell that is not really headless friendly: http://msdn2.microsoft.com/en-us/library/ms913692.aspx.

Friday, April 20, 2007

XPe tip #31: Wallpaper change

Want to change the wallpaper of your Desktop? Just follow this procedure:

The registry entry you want to preset in TD is:
[HKEY_CURRENT_USER\Control Panel\Desktop],"Wallpaper"=Reg_sz:"path to image"

To affect all users:
[HKEY_USERS\.DEFAULT\Control Panel\Desktop],"Wallpaper"=Reg_sz:"path to image"
Note: This entry will also be used as wallpaper in Winlogon time.

That registry value can be overwritten in the registry section of "Win32 API - User" component.

Thursday, April 19, 2007

MEDC 2007 is coming..

MEDC 2007 is coming! I am anxious to go there this year and hear the latest on XPe and CE front. So much info and so little time though.
Sessions info is looking very encouraging and interesting.

AutoPlay without Explorer

I am filing this not as a tip but rather as an encouraging info. Unfortunately, I cannot reveal any technical details of my implementations here at this stage of the project (done for a customer) but more to show up soon, I hope.

It has been a problem for quite some time to get some of the Explorer-only features working on XPe without Explorer shell. Myself I had a negative post to the newsgroup on the subject.
The real challenge appears there not when you remove the Explorer.exe from the image but when you stop launching Explorer as shell.

Removing Explorer gives you a lot:
- Explorer is in general a big security hole.
- It is hard to get rid of all unwanted pieces of the Explorer.
- Explorer is not for embedded (it is not "embedded" type of application and have never been really componentized).
- Explorer increases your image footprint. Explorer dependencies are awfully big.
- Explorer implements only 2 foot UI and it may be very inconsistent to your own applications GUI.
- so on.

Anyway, I finally managed to develop my own shell that implements missing pieces that were only working with Explorer Shell so far. Someday I will post what are the missing features among which is the AutoPlay. But the important message now is that AutoPlay can be working without Explorer shell running in background. You just have to do some hacking and coding within your own shell app.

Another good news is that I got the same shell app up and running on Vista (with some mods though) where I stripped out Explorer components.

XPe tip #30: Where is intl.cpl?


Surprisingly enough but this question keeps coming up in the newsgroup.

The answer is simple: beside the "Regional and Language Options" component that you might have already added to your configuration, please also add "International Control Panel" component that is located under "Software : System : User Interface : Shells : Windows Shell" category.

XPe tip #29: How to prepare and add Group Policy at build time

We all know how Group Policies are helpful in XPe world. It is not a problem to edit the policies at run time with GPEdit (Group Policy Editor) or directly in registry. But who want to repeat the same procedure many times while deploying images during development time?! So, this needs to be somehow automated.
Also, not to mention that the GPEdit with all the dependencies it requires is quite heavy.

But how to set up the policies you want in TD at image design time?
A few way to accomplish that:

- Mess with the Group Policy INF files and get them into your image updated. This is typically more complicated way because it is often not clear how policies described in the INF. Also, you'd need to bring GP updater tool as well.

- Set up most of the policies you need as registry entries and add them in TD via your own component or TD Extra Registry section.
Here you will find a helpful page (Group Policy Settings Reference) where you can download Excel document with most of the Group Policy listed with corresponding registry entries mentioned.

- My favorite. I have provided this answer in newsgroup a while ago.
Set up policy settings on XP Pro machine or on your own working XPe image that includes GPEdit tool and then save the following folder with all its content:
%systemroot%\system32\GroupPolicy.
This folder contents MACHINE and User specific policies set up. Those are just .POL files. It also contains the shutdown/logon/logoff scripts (only if you didn't change the default location for those in GPEdit).

XPe tip #28: Boot.ini Extra Switches


How do you change boot.ini for an XPe image during build time? The answer is - manually :-(

TD does generate boo.ini for you with the default ArcPath based on what you set up on the Configuration page. It is done by the EndOfBuild event handler of the Base Component that basically contains the Configuration build logic. However, if you plan to play with dual boot or want to change a logo or add some other boot.ini switches you have to modify just built boot.ini manually every time you do a build in TD. This is typically not a big deal but annoying, especially if you forgot to change boot.ini just before you deploy the image.


There was a few requests in the newsgroup for a way to change boot.ini content directly in TD. E.g., this one. For some reason Microsoft hesitated to implement that (I suspect the feature got a low priority) so I finally decided to step in and do something for XPe devs in this area.

Since TD doesn't support plug-in interfaces I could only implement that feature through undocumented CMI interfaces. Moreover, I didn't want to rewrite the Base Component (possible and I've done that once for debugging purposes) I decided to add functionality that allows us to add extra boot.ini switches. After all, we could always change ArcPath on the Configuration settings page but never were able to change, say, boot loader logo (the famous bootlogo switch).

I ended up with a script that I added to my XPeTools package that does what I described above. Via an additional field of XPeTools macro component's settings page you can add extra switches to the default ArcPath of boot.ini.

Note: added boot.ini extra switches are persistent and won't be wiped out when you start another session of TD with the same project (or simply close and re-open the project in TD). This is done the same way as I did that for other components of XPeTools package. They all have some advanced component properties used as a storage for the component functionality required settings.




A screenshot of TD with the boot.ini extra switches page open is attached.

Saturday, April 14, 2007

XPeTools component update

Thanks to Jim, a documentation bug was discovered in the DependencyExplorer component.
I now had some time to go over that bug and fix it in the new version of the component.
I uploaded the updated package of XPeTools on www.xpefiles.com. Updated package also includes some bug fixing for XPProEmulation component in UI and script areas. From UI I have removed the leftover Active button, assigned name to the Group filters to remove the confusion caused by the undocumented exclusion filter, fixed other minor UI issues.

Friday, April 13, 2007

XPe tip #27: PID - how to use?

This is an old topic but unbelivably comes online once in a while.

XPe devs should understand the difference between runtime licenses and licensed XPe Toolkit.
Runtime license is provided to you after you signed a legal agreement with Microsoft (through XPe distributor) called CLA. Then you are given stickers with PID numbers on it (numbers like xxxxx-xxxxx-xxxxx-xxxxx-xxxxx). Used to be that the stickers all had different unique PIDs, nowadays they are the same in one multiple-license volume purchase. Depending on how many licenses are covered by your agreement you should be getting some amount of stickers. You use just one PID from any sticker and type it in the Runtime ID field on Configuration Settings page in TD. This will get rid of the "Evaluation build ..." text at run time (it other words, your runtime will be properly activated).

Another story with the toolkit. To be able to create activated runtimes, to even see that Runtime ID field in TD, you must be having and using the licensed toolkit. In Eval toolkit (the one you can download from public MSDN or receive on Eval CDs) that field is hidden, thus not allowing you to create activated runtimes. The CD-key you are getting from Microsoft (aka the toolkit Serial Number) is to be used once when you are installing the tools (TD, CD, database, etc.). Don't put this number in the Runtime field in TD - it is a wrong place for it. Rather type in there a PID from a sticker.

Also, just FYI. In SP2 times there was a QFE issued by Microsoft that was patching the toolkit to be able to use new PID range in SP1 database. That QFE was to address Extend Runtime Product Key Range for Windows XP Embedded. Quote: "This update allows OEM's and embedded developers to use runtime product keys provided after the release Windows XP Embedded with Service Pack 2 on a database containing Windows XP Embedded Service Pack 1 or earlier component databases.".

If you contact your XPe distributor, they should be able to explain you all the details about PId and etc.

XPe boot-wish-list

To continue my post about MBR problems.

I believe XPe documentation doesn't describe this topic in the details it should have to. In some places in the documentation they assume you've got XP Pro running on the device. Which means you've already properly partitioned and formatted the local storage to boot from using XP Pro Setup.
Below is my personal wish-list about what should be included in the next Service Pack for XPe if Microsoft will release one:
- Create one tool (better Win32 app that we can launch from WinPE, XP Pro or any other flavor of XP and nowadays Vista) that will properly partition local storage on target device. After all, XP Pro Setup does that *much* better and more reliable then any tools from XPE toolkit.

- Provide more detail troubleshooting section in the documentation that will cover as many failed case as possible - from broken MBR to boot sector, from changing timeout of boot.ini to debug parameters for KD.

- There should be documentation section (part of XPe docs!!) that explain how BIOS loads the MBR, info about MBR structure, boot sector, boot sequence of XP, loader (how it gets loaded by bootsector) and etc. Ideally they should include info about all the fields of MS version of MBR and boot sectors. After all, most of this info is out there in MS KB articles (but hard to search) and on the Web.

- Provide a set of tools (my best wish is to have one complex command line tool with a bunch of switches like myu favorites MbrFix and NirCmd) that will do fixing for all the parts that may get broken - from MBR to bootsectors (all types!) and partition setup (I secretly don't get why we should be using 3rd party tools like MbrFix, for instance?).

I will probably update this list once in a while when my memory flashes out with more ideas I've been having.

XPe tip #26: Msvcr80 in XPe world

The newest Visual Studio (VS2005 and next version Orcas) by default will link all C/C++ applications you create with the IDE wizard to the latest CRT library which is msvcr80.dll. By default, again, it uses a shared library link (dll must be present in the system path on the target).

Only XPe FP2007 brings in the library but as a part of the huge .Net 2.0 Framework component. It is not even wrapped into a separate primitive component. Obviously, you don't want to add such beast as .Net 2.0 to just add msvcr80, unless you need .Net 2.0 in your image.

Here is a few tricks you can use to get rid of possible problems with missing msvcr80 at runtime:
- Copy msvcr80.dll along with msvcm80.dll and msvcp80.dll libraries from VS2005 folders on PC where you run the VS tools (typically somewhere under "\Program Files\Microsoft Visual Studio 8\VC\redist\x86\Microsoft.VC80.CRT").

- Make sure you are working with the proper version of the libraries corresponding to the project settings - Debug or Release. Often, people are working with Debug build (which is default by the wizard, btw) that requires debug version of the msvcr80 - msvcr80d.dll.

- Please copy manifest built along with your application exe in VS2005. This will make sure it will use the right dependencies versions.

- Often (depends on your application logic and implementation) you can simply use only basic Win32 API and you may not need the CRT libraries at all. The MSVCR80 may get linked however if you have "Multi-Threaded DLL (/MD)" or "Multi-Threaded Debug DLL (/MDd)" option set for the Runtime Library on the Code Generation tab in the C/C++ branch of the Project Properties in VS2005. Just use the single-threaded Dlls there and disable Exceptions. Or, alternatively and my favorite, just link statically to the library to remove the dependency at all (use "Multi-Threaded (/MT)" option for instance).

- Make sure to include side-by-side components in TD.

- If the above doesn't help, install VS2005 (x86) Redist package at runtime.

- If the application still fails to load, run the application through DependencyWalker on the target devices. More precisely, run it through Profiler of the DependencyWalker. Then you should be able to see why it's failing to load.

XPe tip #25: Dial-up autoconnect

Adam has recently posted a link to a nice dial-up tip in his blog that I commented a little bit. Thought I would mention the fix in my blog as well but all credits should go to Adam.

To get rid of the annoying Dial-up connection dialog appearing every time an application is calling to create an Internet connection through WinInet API (the exact component responsible is rasdial code) all you have to do is to set the follwoing key below:
[HKEY_CURRENT_USER\RemoteAccess\Profile\<ConnectionName>],"AutoConnect" =dword:0.
Where the ConnectionName is the name you assigned to the dial-up connection created.

This and more info you can read about dial-up autoconnect in this MS KB article.

I worked a lot on the rasdial components in WindowsCE a while ago but on XP those are a bit different structured and, obviously, without the source code you are provided with less control on XP over the RAS than on WinCE.

XPe tip #24: CSDVersion

On versions of XPe toolkit (database) prior to FP2007 update there was a problem related to version info that gets set in the system registry hive and gets returned by APIs like GetVersion[Ex]. Some applications code or application installation scripts make queries for the system version info to either visualize it to end user, change install path or setup or change the application logic if not quit automatically on unsupported OSes.
Obviously, if the version info is set incorrect those application will fail or won't work as expected (as tested on regular XP Pro with the same SP level installed).

The system version is stored in the CSDVersion string registry value under [HKLM\SYSTEM\CurrentControlSet\Control\Windows] key. You can easy find out what the permitted values are but instead just simply match whatever you've got on the same level SP XP Pro machine under the same key.
Some of the right values I mentioned in this thread.

XPe tip #23: Account password expiration

I hate the account password expiration feature on XP. But on XPe image it is even worth because your users may get stuck there forever if their account is expired. Giving the fact that your locked down image may not allow to fix the problem with GUI, the person who is administrating the target devices will hate you even more.

So, how to get rid of the account password expiration? Actually, easy. A few way you can accomplish that depending on what's included in your image already:

- Use GUI User Manager at run time to change that flag.

- Run the following command at run time:
net accounts /maxpwage:unlimited
You can automate and launch it during FBA with FBA Generic command, for instance.

- My favorite: use command line version of User Manager - usrmgr.exe tool from 2K or XP Resource Kit. It supports a command line argument to change the password expiration on selected accounts. E.g.:
cusrmgr.exe -u UserName +s PasswordNeverExpires.

-Use Group Policy (GPO) template to reset the security settings of password expiration timeouts. Here is a nice MS KB article explaining how to do that. The option you are looking for is MaximumPasswordAge under [System Access] section. So, you create a similar INF file, where you set the MaximumPasswordAge to -1 - unlimited, and install it during FBA using secedit.exe tool (Security Configuration Engine Command Line Utility component). Here is a simple INF you could create to get rid of the annoying password expiration feature:
[Version]
signature="$CHICAGO$"
[System Access]
MaximumPasswordAge = -1

XPe tip #22: Disk read errors

Perhaps, most of us who ever tried to partition a drive manually (not using XP Setup CD) have seen a boot time error where it says "disk read error" or something similar coming from BIOS.

There was quite many thread about that in newsgroup. Here is one of the latest.
I am not going to go deep here and explain why it might have happened if you used Microsoft tools like diskpart and etc. Just keep in mind that tools like diskpart are intended to be used by IT technicians who know their stuff and dealt with similar partitioning problems many times. Now you also do :-)

I found it extremely easy to repro the issue if you use VPC (Microsoft Virtial PC 2005-2007). Just create a new image there and partition the drive with diskpart. Even if you set the system partition active, you may end up with no booting image because of the screwed MBR record. This of course tends to be the case from machine to machine. I mean it is not 100% reproducible if you move to another PC.

The fix is actually quite simple. I posted it many times:
- Mark the system partition as active if you haven't done so.
- Try fixing MBR. You can do it manually (just have to get used to MBR binary flags and partition table settings) or you can use MbrFix tool I mentioned in the newsgroup and in this blog. MbrFix tool is lovely and provides lot more functionality.

XPe tip #21: Dependecy Explorer vs Configuration Explorer

There was recently a bug found in the documentation of the Dependency Explorer component that led to some confusions I had to remove in the newsgroup.

So, what's the difference between Dependency Explorer and Configuration Explorer components.
The DependencyExplorer component is aimed to walk through components in the database. While the ConfigurationExplorer component serves the purpose of walking through components in the Configuration.

If you fire up Settings page of the DependencyExplorer component and type in a component name in the search edit box and hit Apply button you will see the following sections listed in order:
"Components:" section lists components that comply to the search criteria. Even if you typed in an exact component name, you will still likely see a few components listed here as there might have been a few revisions of the same component imported in the database and released.

"Component Dependencies:" section lists components that the component selected (clicked on) in the above section depends on.

"Component Chain:" section lists components in the inheritance chain of the component selected in the first section.
Note: the inheritance chain has nothing to do with the dependency chain.
Here you're supposed to see all the base prototypes of your components, if any, up to the very root one and the component itself.

"Custom Properties:" section lists advanced properties of the DependencyExplorer component (not relevant here).


Now about the ConfigurationExplorer Settings page:

"Configuration Components:" section lists components that comply to the search criteria and belong to the config. Here, if you typed in an exact component name, you will likely see one component listed as it is hard to add a few revisions of the same component to the same configuration (but not impossible!).

"Component Dependencies in Configuration:" section lists components that the component selected in the above section depends on. Only those dependencies listed here that are included in the configuration.

"Configuration Components depend on the selected component:" section lists components that depend on the component selected (clicked on) in the first section but only included in the config. This is similar to the result of the DEPTREE command of XPECMD tool released by Microsoft with FP2007 but not exactly the same. The difference is that the DEPTREE walks through the database while the ConfigurationExplorer also filters out components to list only those that belong to the same Config. If you need the DEPTREE functionality there, you just use ConfigurationExplorer component within a huge config like XPProEmulation project.

"Component Chain:" section lists components in the inheritance chain of the component selected in the first section. Here again you are supposed to see base prototypes of your components, if any. The prototype components are all listed here, regardless of if they all belong to the Config, and the component itself.

"Custom Properties:" section lists advanced properties of the ConfigurationExplorer component (not relevant here).

Now let me show you the results you actually get from if you explore XPProEmulation component within XPProEmulation project using both tools.

DependencyExplorer:
Search box: XPProEmulation

"Components:" section:
XPProEmulation [Version ...], [Visibility=1000]

"Component Dependencies:" section:
No component dependencies found.

"Component Chain:" section:
XPProEmulation [Version ...], [Visibility=1000]
Default Prototype Component [Version ...], [Visibility=1]


"Custom Properties:" section:
....


ConfigurationExplorer:
Search box: XPProEmulation

"Configuration Components:" section:
XPProEmulation [Version ...], [Visibility=1000]

"Component Dependencies in Configuration:" section:
No component dependencies found.

"Configuration Components depend on the selected component"
XPe Tools [Version ...], [Visibility=1000]

"Component Chain:" section:
XPProEmulation [Version ...], [Visibility=1000]
Default Prototype Component [Version ...], [Visibility=1]


"Custom Properties:" section:
....

Hope this clears things up.

XPe tip #20: XPe Tools package

I keep mentioning XPe Tools package from www.xpefiles.com in newsgroup.
So, what is it? It is a set of tools I created for XPe developers to help exploring components and configurations, upgrading to newer version of SP, fighting with component dependencies and so on and so forth.
Actually, the initial purpose I created the first component in the package (XPProEmulation) was to get familar with undocumented CMI interfaces that XPe toolkit uses. Some day I'll write a few notes and hopefull articles here in the blog about CMI and what can be done using it.

The tools are implemented as components themselves (I thought it would be much more useful to get your hands on the tools by not leaving TD environment). More or less TD plug-ins, although TD lacks the real plug-in interface.

When you import the XPeTools.SLD, you will get a few components added to the database and they will appear under "Software : Test & Development" category in the Component Browser in TD. No repository is provided or required.
The new components you are going to see are XPe Tools (macro), Configuration Explorer, Dependency Explorer, Derive Explorer, XPPro Emulation and Smart Upgrade. Every component from the package has a link to an encrypted CMI script and DHTML page representing that component functionality.

You may include all the components into your Configuration, the image won't grow in size since no files added, by resolving dependencies of the macro component "XPe Tools". Please use XPeTools Package that includes all the components mentioned above to operate (delete/add/etc.) on the package components at once.

You will get more details on what components do and how they are implemented if you look at the Readme.txt file provided within the same zip as the SLD file. Also, all the components have links on their Settings pages ("more info") to show you more information about the current component and functionality.

I will briefly describe here what components are and what they do:

XPeTools - macro component that brings in all other components from the package. Some useful global options and flags added. Boot.INI extra switches.

Configuration Explorer - this component allows you to navigate through Configuration components to see their dependencies and components that depend on them. It also allows you to save configurations in plain text files (useful for further project comparisions).

Dependency Explorer - allows you to navigate through XPe database components to see their dependencies. Variaty of component filter options is available.

Derive Explorer - allows you to navigate through XPe database to see derived components. (got to get this worikng on FP2007)

XPPro Emulation - allows you to add group filtered components from XPe database to your Configuration. Inclusion and Exclusion Group Filters are suported. Due to some bugs of SP2 FP2007 (me reported to Microsoft) the exclusion filter is now required in order to get rid of Hardware components from the final Software-only XPProEmulation project.

Smart Upgrade - allows you to upgrade components in the Configuration preserving their extended properties. Smart Upgrade component will allow XPe developers smoothly "port"their SLX projects to SP2 and later preserving the configuration component settings during the upgrade (the settings of the "User Interface Core" component, for instance). This component should work for SP1-->SPx ports as well as for RTM-->SPx.


One quick comment about one option of the DependencyExplorer and other components - Minimum Visibility. This option is very helpful in the filtering feature of the tools. But I mainly use it to lower the visibility below 100 as the TD doesn't really allow that (well, it does but it requires a reg hack that I will post some day).
And please don't hesitate to lower the option value down to 0. Then you will be able to see even Legacy components left in the database from previous version of the product (and still distributed by Microsoft). you should not bother to add those components to your configuration(s) but it is a great knowledge you can gain if you explore those components as well.

XPe tip #20: Active HTML Desktop

An interesting question recently popped up in the newsgroup about how to get a toolbar working in Active HTML Desktop on XPe.

As you know, on XP if you wanted to set up Desktop to show an HTML page you'd go to -> Properties -> Desktop -> Customize Deskop -> Web page and add a web page (aka component) there with soem setting options assigned. When it was done, your Desktop would show the selected HTML page(s). I always use this feature on my XP machines where I set the IE Home page (typically Google.com) to be the Desktop.

This feature works ok on XPe, of course. However, an issue was reported about the toolbar of the Active Desktop not being popping up.
On XP, that toolbar (a popup that appears on the top of the Desktop when you are hovering mouse cursour over the top of the desktop HTML page) gets disappeared when you have the desktop components locked. As I mentioned, components include the web pages you set for the active desktop. The lock can be set with GUI - just open up Desktop Items dialog (Display -> Properties -> Desktop -> Customize Desktop -> Web) and clear the "Lock desktop items" checkbox at the bottom.

But we are always interested in the registry entries mapped to particular XP features when we have to pre-set those in TD for XPe builds. The HTML Desktop Lock is mapped to
[HKCU\Software\Microsoft\Internet Explorer\Desktop\Components],"GeneralFlags" dword registry value. It is a bit mask and corresponds to a few general options of the page display.

If it is set to "2" (or better say, the second bit set) the desktop items are locked and the toolbar won't be sliding out. You should reset it to 0 if you want the toolbar to appear by mouse hover event.

I am going to update this post with some more information as required to clarify a few things about the ActiveDesktop feature of XP.

First of all, the feature works great on XPProEmulation image. Note that to get it to work there didn't require any custom registry entries to be populated.
A few more tips here:
- Shell32.dll must be in the image. Well, since the feature doens' make sense out of Explorer Shell - Desktop is required - it is almost useless to say the Shell32 is required too.

- Make sure "Primitive: Browseui", "HTML Rendering Engine", "Primitive: Oleaut32" and "Primitive: Shlwapi" components are in your image. Last 3 components are probably in your image already if you resolved Explorer Shell dependencies.

- Make sure the user you are logged in have Admin privileges.

- Check if Desktop.htt file gets generated under <%UserProfile%\Application Data\Microsoft\Internet Explorer> folder.

- Make sure "Active Desktop Mover" object is registered. It will be mentioned under [HKEY_CLASSES_ROOT\CLSID\{72267F6A-A6F9-11D0-BC94-00C04FB67863}] key. Basically this object is what you see as the toolbar when you move mouse cursor over the top area of the page.

- Make sure the Low security settings for My Computer (local) Internet explorer zone are set.

To explain you why I ask you to check the above, I will have to explain you how the ActiveDektop feature works with a Web component setup. When you set it up in the Desktop Items dialog, Explorer Shell automatically generates an html based file - Desktop.htt - under a user profile folder (the one mentioned above). That HTT file is the Web page you see on your Desktop afterwards. It may contain a few components (Web pages) - they are all FRAMEs within the HTT html file (tag IFRAME used). The HTT page also has an object embedded on the page (OBJECT tag used) - Active Desktop Mover - that is basically the toolbar you see when you move the mouse cursor over the top side of the page. All the Web components and the toolbar appearances are controled by a set of values and subkeys under [HKCU\Software\Microsoft\Internet Explorer\Desktop\Components] key.

Please note that you can open that Desktop.htt file within IE as a regular dhtml page - you will notice it looks very like your Desktop. Obviously, there is no default association for .htt file on XP(e) so you will have to either manually associate it with IE or simply launch the following command line:
iexplore "%UserProfile%\Application Data\Microsoft\Internet Explorer\Desktop.htt"
(replace the %UserProfile% with the actual value of the env.var before the launch)

When launched, you will have to allow IE to render the page with the ActiveX control embedded (the Active Desktop Mover control) via a little popup appearing as the IE info bar.

All the above applies to IE 5.5 and any later version.

What launch in IE gives you? A perfect tool to test whether the toolbar appears or not when you move the mouse cursor over the top of the page. You can fire Regmon/Filemon (better Process Monitor) while running IE instance and monitor what's going on there when you navigate the browser to the HTT page. It will show you all the object Dlls loaded, registry access, etc. Or, better, run the above command line and the mouse hover test through DependencyWalker Profiler (open \program files\Internet Explorer\iexplore.exe, hit F7, type in the full path to HTT file and hit OK). It will show you in red if any Dlls is missing while you are testing the page.


Last update on the issue. When you target the feature for your XPe image and don't want to use a full blown image like XPProEmulation, please make sure you include the template for Desktop.htt. The template is going to be used by the shell to generate the actual Desktop.htt with the list of components you place on your Desktop. If you don't include the template in your image, Web Desktop feature implementation code will use a dummy HTT file that won't have a reference to the toolbar ActiveX component and therefore won't show it on mouse hover event.
Easieast way to get the HTT file in your image is to add "Images and Media" component. However, the component is quite big so you may end up just adding the HTT file only via your own component or TD Extra Files section. That file is is the XPe Repository anyway.


Good luck with debugging Web components on your ActriveDesktop!

XPe tip #19: Microsoft Access database

How to get an Access database working on your image with ODBC? Answer is here.

Basically you need to include "Jet Database XXX" components in to your image config. Particularly the "Jet Database ODBC Support" component.

"Microsoft Data Access Components (MDAC)" component with all dependencies is also required.

XPe Lifecycle

I thought Microsoft has covered the life cycles for their products well enough but questions about it keep coming. A recent thread in the newsgroup is a good example.

Very details answer to this was given out by the Product Team in their blog - XP Embedded Lifecycle Support.

Some more helpful links on the subject.

Life cycle of Service Packs for Microsoft Embedded products is detailed here.
Here is what it says about Windows Embedded:
Product Name General Availability Date Service Pack Support Retired Notes
Windows XP Embedded Service Pack 0 (RTM) 30-Jan-2002 22-Oct-2004
Windows XP Embedded Service Pack 1 22-Oct-2002 10-Apr-2007
Windows XP Embedded Service Pack 2 18-Jan-2005 Not Applicable
See Note Support ends either 12 months after the next service pack releases or at the end of the product's support lifecycle, whichever comes first. Visit the Lifecycle page to find the support timelines for your particular product.

Microsoft Support Lifecycle page for Windows XP Embedded.

According to Shayna Begun [MS], Feature Pack is not the same as a Service Pack - Feature Pack 2007 did not reset the support lifecyle for XPe. Regarding the next version of the product, it would be the next version of the Embedded product, so Vista would not pertain here.
Andy Allred [MS] confirmed that as well here. (you can find the whole thread here)

Just thought I'd gather all the related info in one place.

XPe tip #18: Hibernation doesn't work

Things that can prevent hibernation from working on your runtime:

- Some driver conflicts or failures. The best way to fix these is to check the Device Manager to see if any exclamation marks there. Command version of the Device Manager - devcon - can also be used here. However, I personally prefer FBALog.txt and setupapi.log log files that are created under the %SystemRoot% and %SystemRoot%\FBA folders respectively.

- None or a wrong display and GART drivers installed. The same tip here - check the setupapi.log log and FBALog.txt files to see why the display driver, if was included and/or componentized for the platform, failed to install.
Note: Standard VGA driver, that takes place when you don't have native hardware driver installed, doesn't support hibernation.

- Some driver states are set to disallow the system going into standby mode. Easiest way to check this is to open a driver properties in Device Manager.

- Not enough disk space for hibernation file. Basically check here if the hiberfil.sys file is getting created on your target system partition.

- Hibernation is not enabled or you are using wrong HAL component. (Check either in TD or with Power CPL at runtime). In TD the hibernation can be enabled on the HAL (PC) component settings page - "Enable hibernattion support" checkbox under "Power Management Settings" section.

- Sometimes the "Disk Dump Drivers" component required. (The diskdump driver takes its part instead of scsiport driver when a RAM dump function is requested. It is just limited version in functionality though.)
Although from my earlier thinking the driver was required for the hibernation to work but in the newsgroup it was reported it is not. So, perhaps, it is only needed for the crash memory dumps to work. It doesn't hurt to add this very small driver component, though.

Sunday, April 8, 2007

XPe tip #17: DST and EWF

Pretty old topic but still no really good solution or workaround exists there on how to make DST working properly with the EWF enabled.

There has a bunch of suggestion been on how to make those two live together posted in the newsgroup: from using undocumented system's RealTimeIsUniversal flag to making your own time monitoring application that will switch or correct the system time if a DST time switch happens.

Perhaps, the only workaround that would work correctly is to disable the DST. Well, I can barely call this a workaround but at least with the DST off you won't have issues with the time changing unpredictably.

How to disable DST? Easy, just set this registry flag at run time:
[HKLM\System\CurrentControlSet\Control\TimeZoneInformation], "DisableAutoDaylightTimeSet"=dword:1

On related note, you can mess around time zone settings offline in TD if you explorer the following key:
[HKLM\System\CurrentControlSet\Control\TimeZoneInformation].
More info about this can be found on this XPe tip here.

Friday, April 6, 2007

XPe tip #16: Can be Eval SLX re-used?

Recently I answered the following question in the newsgroup: is it possible to re-use SLX project created with eval version of the Toolkit with purchased copy of the tools?

We all know that there is no (easy) way to re-use database of eval installation with the full version of the toolkit. If you use it, you are going to end up with eval runtimes.

However, using the SLX (original project) create with eval version with either eval or non-eval versions of the toolkit is perfectly fine as long as you follow the following rules:
- You install the full (non-eval) version of the toolkit properly. You will have to completely uninstall eval version first including database.
- You re-import all the custom SLDs you've created to support that SLX project in to the non-eval database.
- You type in the right PID number in the RunTime field on Configuration Settings page of the loaded SLX in non-eval TD.

Wednesday, April 4, 2007

XPe tip #15: More useful tools..

Continue finding good stuff on the NirSoft site.

Not sure how many times I had to use "at" command on embedded. Especially when I had to launch a command line within LocalSystem context (/interactive switch). Although always had to calculate timing and do a few additional commands in order to schedule and then delete the properly configured "at" command. Now I won't have all this problems with another NirSoft's utility - AtNow

Some good stuff for doing Window sniffing - from class names to style/ex-styles and etc. Two really nice utilities - WinListener and WinExplorer. The second is really good as it allows you to modify all the windows parameters via GUI instead of messing with the code and SetWindowsPos API calls.

Tools like ExeInfo and DLL Export Viewer are also nice. But there are lots of similar tools on the Web. Also, I am a fan of FAR Commander and a few plug-ins there do similar PE analisys and produces even better, more detail and visual results.

In driver and service area - ServiWin. Not as good as devcon but nice GUI makes it easier to see and filter out specific driver services. Also, it shows dependencies which is realy helpful on XP Embedded.

The CurrProcess tool is an extremely simplified clone of ProcessExplorer from sysinternals. Advantages - small and requires less dependencies in the image.

ActiveXHelper could be considered as an embedded [read: simplified] version of Microsoft OLEView tool.

DumpEDID and MonitorInfoView -interesting tools. Don't yet know how I am going to use them but I will.

USBDeview - useful, no doubt. Especially the stop/disconnect function. Although if I needed more info on a particular USB device or even a deep look into the interfaces and protocols exposed by the device, I would use an USB Analyzer.

ShellExView - could be useful when dealing with Shell extensions and namespaces issues.
OpenedFilesView - I am probably going to switch to this from similar sysinternals tools.


Many, many more tools there. Explore the utils part of the website by yourself!

XPe tip #14: The best command line ever

While googling for some API call info I recently hit one website - NirSoft - that had a lot of nice tools and source code very useful for hacking and etc. We all know that we often have to do reverse engineering on embedded in order to understand, use or fix some system components functionality.

I am not going to talk about all the tools on that website but just one that made my day - NirCmd. Note: it is freeware! So much functionality in just one relatively small utility.

I have written myself tons of tools and utilities for embedded, even merged some of them. But never had so many functions implemented in one command line application.

The function I personally loved is - changing display settings on the fly. You just pass in the resolution and bit rate and you are done. Best of all, in some intermittent version they added refresh rate parameter.

Comment: the tool works great on Vista as well. This makes my life on Vista MCE much easier. Basically I can play with any TV (SD, HD, etc.) and have no wear the resolution rate is not supported or, on embedded, having no way to fix resolution problems without Display Control Panel applet included in the image.

Another tools that may be useful in changing screen resolutions and, more important, screen refresh rate is ChRes.exe. This utility is much simpler and all it does is calling to ChangeDisplaySettingsEx API. More about this tool you can find here.

Tuesday, April 3, 2007

XPe tip #13: SID issues

We all know about possible issues that duplicated SIDs can create in your system setup and open a security hole on the network [ok, if you don't, read this]. Probably the best explanation to these issues were given by Mark Russinovich and Bryce Cogswell on the documentation page for NewSID tool.

Anyway, I am not writing this to start another thread on the SID duplication problem (if you Google for it, there are tons of similar discussion on the subject). I have recently answered a question in the newsgroup where the poster was surprised by not seeing the SIDs under HKEY_USERS key. It appeared that he was looking for long GUID-like subkeys but was playing with Minlogon based image. Here is the thread. As you may read, the poster was looking for actually Admin or User account SID which is not supported by Minlogon.

User SID duplication is not much an issue in Minlogon environment. However, there is still computer SID. You can find that SID somewhere in SAM hive (HKLM\SECURITY\SAM), typically listed in the list of Members keys. Much easier way to find out that SID though is to run PsGetSid utility from sysinternals.com. Here is where you can download it as a part of PsTools package.

Here is a list of well-known SIDs defined by Microsoft.
The one you're definitely going to catch with Minlogon image is S-1-5-18, a special account used by the operating system (aka LocalSystem account).

For Administrator account you would be looking for something like S-1-5-21-191058668-193157475-1542849698-500 on Winlogon images only. The only bolded part is machine dependent and generated randomly. All user accounts on the same system are going to have incrementally increased four last digits of the SID shown above. (1000, 1001, etc. instead of 500).
You can see the list of SIDs assigned to the user accounts on your system by exploring subkeys under [HKLM\Software\Microsoft\Windows NT\CurrentVersion\ProfileList] key.

And last, the good intro into what SID is on MSDN.

Monday, April 2, 2007

XPe tip #12: SetupAPI - nothing is more helpful than logging!

To continue my previous post to this blog I want to mention another great tool that is always available to us, XPe developers, on target systems. When I say always I assume the right components are included - kernel and "Primitive: Setupapi". But you can barely find an XPe image that doesn't have the last component added.

Anyway, what is the SetupAPI log? It is a set of API functions that a setup application calls to perform application installation operations. The functions are available to develop a setup application that can install any type of application, from driver to user mode app. MSDN has a big and nice section dedicated to Setup API.

Since most of the issues we are fighting with on XPe are often related to components we ain't provided with source code for, especially third party drivers, having at least a way to analyze failures of those components with the logging tool is a great thing.

Anyway, some quick details for those who never dealt with Setu API logs. By Default the log is centrally located under %windir% folder and called setupapi.log. There are some registry entries to set up and configure how you want the logging to be performed. E.g., the most important value for us is logging level. Often the default log level is not enough to find out what's missing or went wrong with the image. So, increasing the level may help to get more information about the failures and possible get on the right track fixing them.

Here is the instructions on how you increase the log level. SourceQuest also has a freeware GUI utility that allows you to modify SetupAPI configuration parameters without remembering all the configuration registry paths. That utility - suplog.exe, which is already there and available for download, aims to simplify the capture and display of SetupAPI logs. BTW, changing most of the settings is dynamic and will be picked up by the new loaded instance of the app being installed and analyzed.

Quick note here. There is a very useful option for SetupAPI available that allows you to direct the logging to debugger output that you can easy capture with DebugView tool from sysinternals.com. It gives you a nice control over the log for your only component that you are trying to install at run time.

XPe tip #11: Debugging - VS2005 and WinDbg

On Embedded devices we are often required to do some debugging various issues from driver install to application flow. With regards to applications and anything running in user mode everything is simple and straightforward - just stick to Microsoft Visual Studio (better the latest and greatest - VS2005 or coming Orcas) with Remote Debugging feature (firewall caveat on SP2). But how to handle driver debugging?

No easy answer there. The only option is probably KD (Kernel Debugger). There are a few debugger on the market (WinDbg, SoftIce, etc.) but I tend to think for XP and later only WinDbg has the real power to keep up with Microsoft changing environment. The latest version (6.6.7.5 to the moment) has lots of nice new features, although for XPe development we probably only need the basics. So, the winner is WinDbg (obviously, only 32-bit version is interesting option for us, XPe developers).

It requires a little bit of effort to setup the debugger to connect remotely your development PC, where you install the WinDbg tools, and target machine. Here is how you can enable remote debugging on SP2 (firewall setup, policies and etc.). And don't forget to setup the symbols.

For XP(e) you'd probably stick to COM or 1394 debugging. For Vista targets I loved the USB 2.0 kernel debugging support feature.

And you can certainly use WinDbg to debug services.

Very nice tutorial and lots of useful info about debugging with WinDbg I happened to fin on my favortie CodeProject.