I’m looking at the computational cost of computing various hashes. Naturally, I want to collect run time statistics on each hash command and collect this metric several thousand times.
The natural choice is to go with time, but I need to use time’s format option to output a CSV output. Sounds easy:
brooks@saosin:~$ time -f %e,%S,%U md5sum .viminfo
bash: -f: command not found
real 0m0.002s
user 0m0.000s
sys 0m0.002s
WTF? -f is an option, not a command! Come to find out, Bash has its own built-in time command which doesn’t have the same options as the GNU time command! So you’ll need to use the full path to the GNU utility. If you don’t know it, use which:
brooks@saosin:~$ which time
/usr/bin/time
Now, let’s try this again:
brooks@saosin:~$ /usr/bin/time -f %e,%S,%U md5sum .viminfo
0cc07f508925f94b18f50166576b83c7 .viminfo
0.00,0.00,0.00
Better!
You can also use -a -o filename to specify where to put that csv output across multiple runs.
The super timeline often exceeds 65,000 rows and is extremely slow in
Excel. To fix this, split the file into manageable chunks.
wc -l filename.csv gives the number of lines in a file.
split -l 65000 -d supertimeline.csv supertimeline will generate
multiple files named supertimeline.00 (01, 02, etc) with 65000 lines
each. -l is the line count and -d tells split to use digits for the
prefix instead of letters (00 instead of AA). The second supertimeline
parameter tells split to use supertimeline as the prefex. Omitting the
prefix (supertimeline) and -d will result in files named xaa, xab,
xac, xad, etc.
Posted via email from brooksgarrett’s posterous
SANS publishes the SIFT (SANS Investigative Forensic Toolkit) Workstation as a VMWare appliance. The environment is impeccable for rolling out a mobile forensics workstation and is preloaded with a wealth of tools. The workstation was created by Rob Lee.
All that being said, nothing is absolutely perfect. A huge drawback to the SIFT workstation is getting disk_stat working so an investigator can detect HPA’s on a suspect drive. Acquiring a disk with an HPA requires additional steps and missing the HPA can mean missing evidence.
When you try to run disk_stat, you are given an error that the libssl.so.7 library could not be loaded.
sansforensics@SIFT-Workstation:/mnt/hgfs/Evidence/CASE001$ disk_stat
disk_stat: error while loading shared libraries: libssl.so.7: cannot open shared object file: No such file or directory
To fix this, create a symbolic link in /usr/lib pointing to libssl.so. You’ll need to do the same for libcrypto.so as well.
sansforensics@SIFT-Workstation:/usr/lib$ sudo ln -s libssl.so libssl.so.7
sansforensics@SIFT-Workstation:/usr/lib$ sudo ln -s libcrypto.so libcrypto.so.7
All done! disk_stat will now properly detect HPA’s on attached drives. Enojy!
Brian Carrier has provided the forensics community with tools that are absolutely vital to open source forensics. One tool I tend to under utilize is sorter. Sorter is used to ‘sort’ files in an image into categories using file headers as the primary resource. Thus the output is a set of text files (“images.txt”, “”archive.txt”, etc.) which details what the files are. This can greatly reduce investigation time if you know what you are looking for (Images? Sensitive documents? Emails?)
Sorter also has the ability to leverage the NSRL (National Software Reference Library) to sort files with known hashes into an “Exclude” category thus eliminating them from the review process and reducing investigation time. As the NSRL is provided (free of charge) by NIST, this process of removing known files from the analysis procedure is colloquially referred to as “De-NISTing.” the investigator can also use custom hash databases to either alert when a file matches (-a) or exclude and ignore a file when it matches (-x). You can use all three options together (NIST NSRL, alert hash database, and exclude hash database).
To make my exclude database, I obtain a freshly deployed machine from our desktop or server teams. I bring the drive into my forensics workstation and mount the drive ro with an attached hardware write-block in place.
NOTE: Though you don’t need a forensically sound image for this process, it never hurts to practice your skills and maintain good habits.
Use MD5Deep to generate a hash list.
md5deep -klr /path/to/mountpoint > ignore_hashes
This will generate your list of known files which can be safely ignored. For my alert hashes, I often create a list of hashes of various pieces of malware.
Once the hash lists (often referred to as your hash databases) are ready, they need to be indexed before sorter can use them. Use hfind to index the hash databases.
hfind -i md5 ignore_hases
The -i tells hdfind that the file it is indexing contains md5 hashes. This command will output a .idx file in the current directory which sorter will use. For sorter to find and use the index it must be in the same directory as the hash database it indexes.
That’s it for now. With this information, you are ready to go De-NIST!
