Saturday, May 19, 2007

XPe tip #36: Changing IP with EWF/FBWF On

There was an interesting topic recently covered in the newsgroup where I had to spend some time to answer the poster questions and got some new knowledge (as very usual while answering other's questions in the newsgroup, btw :-) ) about Microsoft's Registry Filter component.

I won't go into details of that conversation (read in on the newsgroup archive if you wish) but I will just summarize the task, the problem and the solution.

Prerequisites: We have an embedded device running XP Embedded image with EWF or FBWF filter protecting the system partition including system registry hives.

The task: For a person administrating the device (IT, end user, whoever with the sufficient rights to access the network stack on the device) it is required to change system network settings to either switch to required static IP or, if already a static IP used, change the IP address.

The problem: With the write filter protecting the system registry it will be required to commit the entire system registry hive in order to make the IP address change persistent. With EWF it is even worse - you have to commit the entire system volume to accomplish that. The bad part there is that along with the IP changed you will commit all user modifications he might have done on the device and in the system registry. Often, those modifications are undesirable and lead to an unstable device (the beauty of the WF protection of the system volume is getting lost). I am not even touching the sensitive area of viruses and malicious software that a networked device might be exposed to.

The solution: What if we only commit the changes related to the IP settings in registry (what those are? see below...) and leave the system with the fully protected system volume yet.
Yes, it is possible with the component called Registry Filter which has become public within Feature Pack 2007 (there was a QFE for SP2 release that brought in the Registry Filter but that QFE was only available to OEMs).

The Registry Filter was only created by Microsoft to fix the famous issues with Domain Secret Key and TSCAL (license for Terminal Server client). However, the way the component works allows you (undocumentaly, of course) to use it in the purpose of working around some other issues with Write Filter protecting the system volume.

The way Registry Filter works is that it allows you to specify registry keys to make them persistent across reboots while the write filter is ON. Basically Registry Filter creates and initializes a file-backed ramdisk - the hidden system regfData file under the root of the system volume. On that ramdisk it stores files that contain the selected key contents.

My suspicion (as I indicated here) is that the way the \regfData gets away from the EWF protection is through a mechanism similar to the EwfMgrCommitFile. On FBWF, of course, it is much easier to by-pass the protection for the selected file since this filter is file system aware and doesn't require a hack like the EwfMgrCommitFile API.

Now, when we know how the Registry Filter works, lets try to use its functionality to "unprotect" other registry keys we may need to may persistent across reboots with the write filter enabled on the system volume and system registry hives.
Sample registry entries that have to be added to your XPe image at the design time (at run time of design) I already showed in the newgroup. (Note: now we know the time zone info is not the right key for monitoring because of the way Registry Filter gets started - it gets started after the system checks for time)
Here is a sample for reference:
[HKEY_LOCAL_MACHINE\ControlSet001\Services\RegFilter\Parameters\MonitoredKe­ys\2]
"ClassKey"="HKLM"
"FileNameForSaving"="YourFileName.rgf"
"RelativeKeyName"="Software\YourRegistryBranch"

But what are the keys that have to be set for monitoring by Registry Filter to make the IP change persistent?
Initially I thought these:
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip\Parameters\Interfaces\{}],"IPAddress"
and
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\{}\Parameters\Tcpip],"IPAddress"

But after I've done some tcpip driver stack debugging it is clear to me that the first key is the important one. The second will be auto-reset by the system (tcpip driver will refresh the values there grabbing the values from the first key).

So, cool, this means that we just have to add in registry at run time something like the following and it should work:
[HKEY_LOCAL_MACHINE\ControlSet001\Services\RegFilter\Parameters\MonitoredKe­ys\2]
"ClassKey"="HKLM"
"FileNameForSaving"="TcpIp.rgf"

"RelativeKeyName"="SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\{}"

Note1: The CurrentControlSet is used here correctly. But in case you like the actual ControlSet00X key please make sure you work with the right one that gets used for the hardware configuration you are working with (typically the ControlSet001 on the XPe images by default).
Note2: Replace the interface GUID with the real GUID you can read off the registry at run time and that gets generated unique for your NIC adapter and interface.

The problem #2: No, the above didn't work and that is what the initial poster indicated.
The IP gets changed and the system is ping'able. However, as soon as you reboot the ipconfig.exe tool will report the old IP address that you had before the IP change. While the Tcpip GUI (the UI you get to when you go to the network connection properties) or netsh commands report the new IP. I should also note here that the actual IP of the system that you can feel on the network is still the old IP.
Why?

The upgraded solution: After some debugging I've done for the tcpip stack on XP as well as the Registry Filter and EWF components the issue appeared to be related to the loading order of the filter driver. Let me quote myself from the newsgroup.

First I created an image where I could perfectly repro the problem (Minlogon, netsh, EWF, RegFilter). Then I debugged some system stacks to understand what's going on there when we change the IP with the RegFilter on and set to monitor the keys I mentioned earlier. (I picked up EWF vs FBWF to simplify the debugging, having FBWF there won't change the picture)

I must admit the result was a bit surprising to me. Despite the fact that the behavior of the protected system with IP changed and rebooted was quite strange - no sync in the results from ipconfig.exe tool and "netsh interface ip show address" command (the same as you'd do with TCPIP GUI) - the source of the problem was not in the Tcpip stack or the registry keys picked up for the monitoring. The tcpip driver (or a driver from the ipv4 stack) indeed reads the network adapters info into a buffer (link list) at initialization from the registry's Tcpip\Interfaces key. This, of course, includes the IP address settings. When you do "ipconfig /all" it doesn't really go into the registry again but rather queries the driver for that info stored in the cache (buffer). While the netsh commands are more "dynamic" and will scan and return the results from the registry.

It is important to understand that RegFilter doesn't make the monitored keys transparent to the registry hives. The RegFilter rather applies the new registry values - the values that were saved in the monitored key hives on the protected system volume's ramdisk - to the image's registry hives at the next boot time. Some time during the driver init it loads the ramdisk and reads the files on it. Then it sets the actual values in registry to whatever values it reads from the file contents on ramdisk.

The tcpip driver will populate the internal stack buffer very early during the system boot. Looks like the initialization actually happens before the RegFilter is loaded. This leads to the Tcpip structures populated with the old data but the registry info already reflected the new data (laid out by the RegFilter) after the boot. So we end up with the IP addresses reported out of sync between the ipconfig and the registry.

To summarize the above, the real problem there seems to be that the Regfilter and one of the drivers from Tcpip stack belong to the same driver load group. Without the RegFilter enabled the IP would be changed properly (assuming the change can be flushed). This fact is extremely easy to prove with the following simple experiment.
Forgot for a minute about the RegFilter. In fact remove the Tcpip related registry paths from its MonitorKeys branch, commit EWF and gracefully shutdown the image (now the RegFilter doesn't "unprotect" the tcpip keys). Load up the image SYSTEM hive offline (open it with a regedit on a XP Pro machine). Go to the [HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip\Parameters\Interfaces\{}] key and modify the value of the "IPAddress" property there. Unload the hive and boot to the embedded image and verify that the address has changed and is in sync between ipconfig and netsh results (or GUI). Note that you don't even have to mess with the [HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\{}\Parameters\Tcpip] key as it will be automatically updated during the boot (tcpip driver does that at the init time).

It is also easy to confirm with a hex editor that the ramdisk image from the runtime contains the new registry value (new IP address) but the system registry hive has gotten the old one (and this is ok when you think about the Regfilter architecture and design).

So, what's left to fix the solution we created is to change the loading order of the interfering drivers. We can't mess much around the RegFilter driver - it is very dependent on the EWF/FBWF and, the same time, must be launched early at the system boot. So we only can fix the issue by moving the tcpip driver to load it later than the RegFilter and thus pick up the right values from registry hives.

The easiest way, perhaps, to accomplish that would be starting the tcpip driver by the Service Control Manager (SCM) instead of the kernel. The following value would do that:
[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\Tcpip],"Start"=dword:2.
That will put the driver in the auto-start group (started by the service control manager) instead of the system started (kernel or loader started).
This is a pretty safe change unless you've got some custom drivers in your image that demand the network connectivity (IP) very early at the system boot.

With the changes above committed (if you mess with all this at run time), you can try to change the IP and reboot (without an extra commit) and the IP should be the correct one reported by the netsh (or GUI) and the ipconfig as well. At least this is what I am seeing working here.

Just a side note: Domain Secret Key and TSCAL issues were "easier" to fix for Microsoft with the RegFilter approach since those keys are only read late at the boot process (by Winlogon and Terminal Services components).

14 comments:

Raj said...

This article is very useful to me especially to understand my RegFilter doubt

regards
Raj

Anonymous said...

KM,

I have posted on the XPE newsgroup. I am having trouble getting the above method to work. Is there some method for debugging or trult testing RegFiltr other than trying to change a key and rebooting?

All

I am using FBWF protecting my system drive. I have configured keys in
RegFilter to monitor Reg Keys allowing me to persist a changes in the
IP address and my touch screen calibration.


I read KM's blog article on how to persist the IP address across
reboots but am having no luck


After a reboot, neither changing the IP address from DCHP to static or
static to static (new address) nor touch screen calibration persists.


Can anyone advise how to debug whether RegFilter is working?


My FBWF config is:


File-based write filter configuration for the current session:
filter state: enabled.
overlay cache data compression state: disabled.
overlay cache threshold: 64 MB.
overlay cache pre-allocation: enabled.
size display: actual mode.
protected volume list:
\Device\HarddiskVolume1
write through list of each protected volume:
\Device\HarddiskVolume1: (none)


File-based write filter configuration for the next session:
filter state: enabled.
overlay cache data compression state: disabled.
overlay cache threshold: 64 MB.
overlay cache pre-allocation: enabled.
size display: actual mode.
protected volume list:
\Device\HarddiskVolume1
write through list of each protected volume:
\Device\HarddiskVolume1: (none)


C:\WINDOWS\system32>


MY RegFilter entries are:


[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\RegFilter\Parameters
\MonitoredKeys]


[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\RegFilter\Parameters
\MonitoredKeys\0]
"ClassKey"="HKLM"
"FileNameForSaving"="TCPIP.rgf"
"RelativeKeyName"="SYSTEM\\ControlSet001\\Services\\Tcpip\\Parameters\
\Interfaces\\{69978527-CBF4-48F1-BF1A-54371D56DE3A}]"


[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\RegFilter\Parameters
\MonitoredKeys\1]
"ClassKey"="HKLM"
"FileNameForSaving"="TCPIP2.rgf"
"RelativeKeyName"="SYSTEM\\CurrentControlSet\\Services\\Tcpip\
\Parameters\\Interfaces\\{69978527-CBF4-48F1-BF1A-54371D56DE3A}]"


[HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\RegFilter\Parameters
\MonitoredKeys\2]
"ClassKey"="HKLM"
"FileNameForSaving"="ELO.rgf"
"RelativeKeyName"="SYSTEM\\CurrentControlSet\\Enum\\SERENUM\\ELOSERIAL\
\5&23de782&0&0000\\Device Parameters"


I have changed the TCPIP service startup to "2" as recommended by KM.


THANKS!!

Pierre-François Culand said...

Is there a way to protect the \regfData file against possible corruption in case of a power failure occuring during the write access performed by the Registry Filter Service ?

More about my problem here:
http://forums.microsoft.com/
EmbeddedWindows/ShowPost.aspx?
PostID=2984201&SiteID=47&mode=1

Thank you for any solution and/or advice about this problem.

Pierre-François Culand
www.bobstgroup.com
mailto: pierre-francois.culand at bobstgroup.com

Anonymous said...

Look at this
http://msdn.microsoft.com/en-us/library/bb147518.aspx

Anonymous said...

25 Oct 2007 ... runescape money runescape gold ffxi gil eq2 gold eq2 plat ... eve money cheape isk cheap eve isk buy eve money runescape money runescape gold
...

Anonymous said...

wow gold
wow gold
eve isk
wow power leveling
wow power leveling
wow power leveling
wow power leveling
warhammer gold
warhammer power leveling
bestchina traveland
uggsor
ghd straighteners

Cardi Ugg Boots
Classic Short Ugg Boots

Classic Tall Ugg Boots
Metallic Ugg Boots

Mini Ugg Boots
Sundance Ugg Boots

tiffany Jewelry
















[url=http://www.akgame.com]wow gold[/url]
[url=http://www.wowgoldbank.com]wow gold[/url]
[url=http://www.akgame.com/eve-online-c-8.html]eve isk[/url]
[url=http://www.wow-power-leveling.com]wow power leveling[/url]
[url=http://www.akgame.com/wow-power-leveling-c-9.html]wow power leveling[/url]
[url=http://www.cheap-power-leveling.com]wow power leveling[/url]
[url=http://www.fast-power-leveling.com]wow power leveling[/url]
[url=http://www.akgame.com/warhammer-online-c-64.html]warhammer gold[/url]
[url=http://www.akgame.com/warhammer-power-leveling-c-92.html]warhammer power leveling[/url]
best[url=http://www.topchinatrip.com]china travel[/url]and
[url=http://www.shoppingugg.com]uggs[/url]or

Anonymous said...

Do you know the Atlantica online Gold, in the game you need the Atlantica Gold. it can help you increase your level. My friends always asked me how to buy Atlantica online Gold, i do not know he spend how much money to buy the Atlantica online money, when i see him in order to play the game and search which the place can buy the cheap Atlantica online Gold . i also happy with him.

I do not know why I was very like to play the EVE, so I need eve isk and the eve gold , do you know the eve online isk.

Anonymous said...

runescape gold
runescape money
runescape power leveling

Anonymous said...

wholesale jewelrycostume jewelryhandmade jewelryfashion jewelrypearl jewelrycrystal jewelrysemiprecious jewelryturquoise jewelrycoral jewelryshell jewelryswarovski crystal

Anonymous said...

My friends all told me that the best way to spend rohan online gold is a good way. But I could not like spending my own rohan online crone. rohan crone has many ways for us to use. When you start the Rohan Online game, your character will be level 1. I remembered that when I started playing this Rohan game with some little cheap rohan money. If you do not like upgrading level step by step, you can cost rohan gold to help your character to reach level high.
Due to make cheap rf gold the intense gravity on their home planet, the Bellato are the smallest people. rf gold which in RF Online Game is very popular for many players. Play this online game the premise that we have more enough rf online gold first. They are able to combine creative tools and weapons with some rf money and the Light form of universal magic. Under such sustained attacks they fell from power, yet they have bided their rf cp and time.

Anonymous said...

Once I played habbo, I did not know how to get strong, someone told me that you must have habbo credits. He gave me some habbo gold, he said that I could buy habbo gold, but I did not have money, then I played it all my spare time. From then on, I got some habbo coins, if I did not continue to play it, I can sell cheap habbo credits to anyone who want.

Once I played habbo, I did not know how to get strong, someone told me that you must have habbo credits. He gave me some habbo gold, he said that I could buy habbo gold, but I did not have money, then I played it all my spare time. From then on, I got some habbo coins, if I did not continue to play it, I can sell cheap habbo credits to anyone who want.

Anonymous said...

I enjoyed your post,. I have been wondering about this topic,so thanks for posting. I’ll likely be coming back to your blog.
Video AC Milan|Video Sepak Bola|How To Make a Kite|How to make a origami|Tips for lossing weight|How to six pack abs

Unknown said...

Dofus Kamas|Prix Moins Cher Dofus Kamas|Kamas par allopass|Dofus kamas audiotel|Dofus kamas par telephone sur Virstock.com

Meilleur prix dofus kamas stock de dofus kamas

Prix moins cher dofus kamas
vente dofus kamas sur www.virstock.com

Kamas par allopass sur dofus kamas

Dofus kamas par telephone sur dofus kamas

Dofus kamas audiotel sur dofus kamas

http://www.virstock.com/jsp/comments.jsp dofus kamas vente

http://www.virstock.com

steamer2012 said...

Runescape accounts power leveling Runescape gold,items and hacking