COMMAND
NTFS Alternate Data Streams
SYSTEMS AFFECTED
NT using NTFS
PROBLEM
On the NT File System, a file is represented by a fixed-length
record in the Master File Table, or Mft. Each record in the Mft
contains a number of sections that describe "attributes"
associated with the record. There are different attributes
including "Standard Information", "File Name", "Symbolic Link",
"Volume Name", and "Data" attributes. (This is clearly not an
exhaustive list). Attributes can be either "Resident" or
"Nonresident", which describes the location of the attribute in
relation to the Mft record itself. As one might assume, Resident
attributes are those which live directly within the given Mft
record, while Nonresident attributes live elsewhere on the
physical volume. When an attribute is Nonresident, the Mft record
contains information pointing to the location of the attribute.
The attribute relevant to this discussion is the "Data" attribute.
The Data attribute typically contains information pointing to the
actual data comprising the file. As with the other attributes,
the Data attribute can be either Resident or Nonresident.
The term "Alternate Data Stream", or more appropriately, "Multiple
Data Stream", refers to the fact that a given Mft record can have
more than one Data attribute.
John Hayday made an example. Create file:
echo this is a test file > test.txt
more < test.txt
dir test.txt
You should then record the file content and size. Hide data:
type winnt.bmp > test.txt:secret
more < test.txt
dir test.txt
You should then check the file content and size. No change!
To display hidden file do:
pbrush test.txt:secret
This applies if the binary file is small, but if it is bigger it
won't work. For bigger binary files you should use 'cp' (not copy
as it doesn't work) you can copy binary files without corruption.
Test was by hiding explorer.exe (191 Kbytes) in a 12 byte file:
cp explorer.exe test.txt:explorer.exe
cp test.txt:explorer.exe explorer.exe
then extracting it an running it.
The current problem with streams is that many Windows NT users
(including administrators) are not aware that streams exist and
even if they know of them have no simply method of detecting them.
Bill Potvin made an effort to collect a number of "points"
concerning them (we are talking about multiple data streams or
mdss):
1. Mdss are not new. They have been a part of the Ntfs since NT31
and are described both in Helen Custer's "Inside Windows NT"
and in Volume 1 of the NT35 Resource Guide, page 169.
2. Standard tools which ship with the operating system do not
offer a mechanism for viewing these files.
3. There ARE tools freely available, some of them for a number of
years, which DO allow you to see Mdss.
4. Programmatically, there are two ways to view Mdss:
+a. Use the undocumented NtQueryInformationFile kernel Api.
This Api requires an open handle to the file being
examined. This means that files held open for exclusive
access by either the operating system or other running
programs will NOT be enumerable.
+b. Scan the Mft directly. This is the ONLY way to find ALL
Mdss that might be attached to files on the volume because
it does NOT require that ANY files, except for the $MFT
file itself, be opened.
At this time and to my knowledge, these are the only ways to
acquire Mds information programmatically and both are
undocumented.
5. The Security Descriptor that is assigned to a file is also
applied to any Mdss that might be attached to that file. No one
can do anything with an Mds on a file that they aren't able to
do with the "main" file itself.
6. The CopyFile and MoveFile Win32 Apis always retain Mdss as long
as both the source and target of the Copy or Move operation is
an Ntfs volume. The "Backup" functions are NOT the "only
reliable" way to move them around, however they will retain and
restore Mdss.
7. The DeleteFile Win32 Api operates on Mdss.
8. The times associated with a file containing Mdss are all
updated if an Mds on the file is manipulated.
9. The size associated with a file reflects ONLY the size of the
primary stream. It does NOT include the size of any Mdss that
might exist on that file.
10. If you transfer a file containing Mdss using any method other
than the Copy/MoveFile Win32 Apis, the Mdss are NOT retained.
This means you cannot ftp, (or "http"), a file to your system
and find Mdss on it. The backup/restore issue is not relevant
in this case.
11. You CAN transfer a file containing Mdss to a Network Drive, if
that drive is an Ntfs volume, and the Mdss WILL be retained.
For example, "copy foo.txt \\PHOENIX\D$\Temp\bar.txt" works
just as well as a copy to another file on the local drive.
12. Adding support for Mdss within existing archival programs is
not a difficult task. Mdss are read just like any other file
and could be stored in the archive as just another file. The
most difficult issue is the disposition of the Mdss on non-Ntfs
filesystems. However, if this is relegated to a "user
interaction" issue, i.e. prompt the user for disposition
information, then it becomes trivial. The greatest barrier here
is that functionality to detect the presence of Mdss is
undocumented.
13. With respect to "virus" scenarios, consider the following:
+a. Programs running under Windows NT have their file images
held open for exclusive access by the operating system.
Therefore, a running program's on-disk image can NOT be
modified, EVEN BY ITSELF. The in-storage copy of the
program CAN be modified, IF the prevailing security context
allows, using the WriteProcessMemory Win32 Api. Such
modifications are NOT persistant beyond the given execution
instance and do NOT effect any other instances of the
program that may be running.
+b. Any write performed on a file updates the Timestamp
information for that file. EVEN MDS WRITES.
+c. A review of #10, #11 and #12 above indicates that the
mechanisms for transferring files containing Mdss are
severely limited.
14. Mdss apply only to Ntfs volumes, but regardless of media. If
you can format a VHS video tape with Ntfs, then it will support
Mdss.
15. The Win32 "Backup" Apis retain and restore Mdss, when reading
from and restoring to an Ntfs volume. So, if your tape backup
software uses these Apis, then Mdss are relevant.
SOLUTION
MARCH Information Systems has developed a command line utility
which solves the problem of hidden data by checking a machine for
the existence of non-default streams (a 'data' and 'security
descriptor' stream exists on every NTFS file and directory). The
utility searches an NTFS disc locating and reporting the size
and, more importantly, the name of every alternate data stream
detected. If desired it will even report the sizes of the
standard streams. The free utility, together with a paper giving
further details of the threats posed by streams, can be download
from:
http://www.march.co.uk
Also, Bill Potvin made "dumpfs" utility to optionally scan Ntfs
volumes for Multiple Data Streams and it's available in the Free
area (System section) at:
http://www.merxsoft.com
There are two ways to look for these things. One method requires
that each and every file be "opened", while the other scans by
retrieving FileRecords from the Mft. The problem with the first
method is that you cannot look for Mdss on files that you cannot
open. The second method avoids this problem, not to mention it's
*lots* faster. The problem with the second method is that it's
"more" undocumented than the first since it requires a knowledge
of Mft FileRecords in addition to the Fsctl used to retrieve those
records. Since both methods are undocumented and, therefore,
equally "dangerous", the second method is used for the dumpfs
utility. This means that dumpfs will find Mdss that might be
attached to *all* the files on the volume, regardless of whether
they are being held open by some process or even the operating
system.