File MD5 Checksum - How to tell if a file has changed

File MD5 Checksum - How to tell if a file has changed
File MD5 Report - Batch MD5 Reporting
File MD5 Batch Inventory

A file's MD5 Checksum can tell you when a file has been modified from the author's original version.  For example, when downloading a program from the Internet, developers often will give you the file's MD5 check-sum -- your version's MD5 could be compared with the author's to see if there was a change.  MD5's can be considered as a "thumbprint" -- a summary representation of the file's contents.

For example:  MD5: 9f-46-58-ef-3d-fe-76-45-65-61-f0-d3-a7-f3-62-bc

Seeing a file's MD5 checksum, sometimes called a 'hash,' requires software.  I have written two handy utilities, one of which is a batch/command-line program (this article) and the other is a much simpler MD5 Windows viewer. 

keyliner link:  Windows Midy5 MD5 Viewer

MD5 Batch Inventory Program

This utility can report the MD5-hash for a single file.  But more interestingly, it can keep track all MD5's in a directory -- keeping a mini-inventory.  With this, you can tell when a file has been added, changed or even if deleted! -- regardless of the filename or date-time-stamp.

2021.11 Update:  Updated program to .dotNet 4.8


Problem at the Office:

At the office, I had a directory of photographs that are weekly refreshed by another system (employee photos).  Each week the entire inventory was replaced.  Original filenames and sizes remained the same, but because the files were re-downloaded each week, the date-time-stamp changed and all files had a new date.  I needed a way to tell which files were different. 

This Keyliner utility resolved these problems.  It is able to tell if the same-named, same-sized photo contained different contents, even though every file technically changed.  It can also tell who is new and who has been deleted. 

It works like this:  If the photo were truly changed, it gets a new MD5 (remember, we have to disregard the date-stamp and the file-size)  A one pixel change in a photograph, changing a white pixel to light grey, resulted in this MD5 change:

Original file:
MD5: 9f-46-58-ef-3d-fe-76-45-65-61-f0-d3-a7-f3-62-bc
MD5: 54-d3-9d-69-07-1a-9f-d9-92-1c-1e-2b-54-42-de-a4
One Pixel changed file^

a huge change in the MD5.

Features:

• Show a specific file's MD5 check sum, using this command-line
  
  FileChangeMD5.exe file=filename.ext
   
• Optionally, using a configuration file, report on all file-changes within a directory, showing
  -Changed Files
  -New Files
  -Deleted Files
  
  
• It keeps track of changed files by using a text-based database
• Output is an ASCII tab-delimited file (default name: "Inventory.MD5") 
• Runs in Batch or interactive command-prompt mode
• Runs unbelievably fast; directory is parsed in sub-seconds
   
• Can report, grouping by changed-status, or by filename
   
• Free, no charge, no registration, no nags
• Simple EXE - No installation required
• No registry changes; no databases
   

Example Output:

This shows the inventory of all files in a directory (C:\Temp).  Notice the Status column:  "nochange", "changed", "new", and "deleted".  Each file's checksum/MD5 is listed:

Click for larger view

This file can be consumed by other software, parsing for the Status Column. 

Optional Inventory Reporting:

If the config file's "InventoryReport" field is populated with a file name, a secondary inventory report is generated.  The is a simplified report, showing only the type of change and the file name.

Individual File MD5 Checksum

The utility, without any configuration or setup, can report a single file's MD5 checksum, showing the results to the console. 

This example shows the MD5 hash for the file named "C:\Temp\file-01.txt"

Click for larger view

To view a single file, run the program from a Command prompt.
Use this command: 

filechangemd5.exe  file="C:\path\somefile.ext"

where the keyword "file=" is required. 
use quotes around the path and filename.

Better, use a second keyliner utility, which is a Windows-based program:
Link:  Midy5

Download and Installation

To install this program, download the .exe and place in any folder on your hard disk.  Double-click the .exe to run - no installation required.  This program is a DOS program and would prefer to run from a command prompt.

Since keyliner cannot afford a signing certificate, you will be prompted that the file is not safe (being downloaded from the internet).  Click "more information" and allow the program to run.

Follow these steps for a more professional installation:

Using the .exe from a download folder, or copying to a (my Documents) folder is a quick workaround for various Windows 10 and 11 security concerns.  Some vendors recommend this, but these folders are inappropriate for executable software.  Instead, the program should be copied to Program Files so it gains the protection of other Windows security features. Total time: about a minute.

A.  Download this Keyliner utility, from Keyliner's GDrive:

Link:FileChangeMD5 Download Folder

This utility is free for all personal and commercial use.
No registration.  No logging in.  No email.  No nag screens.

Ironically, it has this MD5: 54-d5-e7-f6-59-7a-96-6b-6c-0e-f8-90-ac-95-b9-5d

When downloading, different browsers behave differently.
You will be prompted the file cannot be scanned.  Click "Download anyway"

Microsoft Edge:

Prompts "FileChangeMD5.exe" was blocked because it could harm your device. 

"Click See More" and allow the download.  With Edge, the file will appear in your Downloads directory with a random name, "Unconfirmed 780359.crdownloaded" (name varies).  

Rename the file to "FileChangeMD5.exe".

B.  Mark the program as safe-to-run:

(This step may not be needed if downloaded by Edge and you clicked "More / Download Anyway")

Using File Explorer,
right-mouse-click the downloaded (and re-named) .exe
Select "Properties"
Check [x] Unblock.  This removes the "mark of the web." 

                 Click for larger view

* Only do this if you trust keyliner *and* only if downloaded from keyliner's public GDrive. 

If "Unblock" is not visible, it has already been unlocked (by Microsoft Edge).
Once [x] Unblocked is clicked, this security menu disappears.

  C.  Create a Program folder to hold the program:

Using File Explorer, Create a working folder
C:\Data\
Create a sub-folder  C:\Data\Monitoring  or any similar folder)

If saving to "C:\Program Files"  or "C:\Program Files (x86)", see below for advanced installation ideas.  C:\Program Files is protected by Windows security.  If you use Program Files, always use a separate INI file -- it cannot assume the current directory.  Details below.  Saving the exe in a less-protected folder (c:\data, or C:\downloads is simpler), but with that said, I run mine from Program Files.

D.  Copy the .exe to a working Data folder:

Using File Explorer,
Copy/paste the .exe from the temporary/download folder
to C:\Data\Monitoring

Do this copy as a two-step, copying from the download folder into a working folder.  For this program, it should not be directly downloaded to C:\Program Files without following the advanced steps, below.  Besides, Windows security will not let you download directly into Program Files (technically, you will not be able to remove the "mark of the web").

E.  Run the program and build a default configuration file:

From a temporary directory:
Click Start, (type, no quotes) "CMD"  (launching a command prompt)
C:  (enter)
CD \Data\Monitoring  (enter) 
FileChangeMD5.exe  (enter)

This generates a default INI file in the same folder.  See file: "FileChangeINI.md5"  (named oddly, by design)  You must have rights to write the file in this directory.

See below for command line parameters and how to edit this file.

F.  Testing:

To see the MD5 of an individual file, type this command, where this command is checking the MD5 of itself:

FileChangeMD5.exe file="FileChangeMD5.exe"

Bulk Monitoring and Configuration:

To inventory a list of files in a directory, and to watch for any changes, edit the default configuration file.  Build as many configuration files as needed, for as many monitoring directories as needed.

G.  Edit the sample configuration file, built in Step E. 

The file arrives in the same directory as the executable.

Default INI file "FileChangeINI.MD5"      

Using Notepad
* Modify "FileDirectory"  from  C:\Temp  to a directory that you want to monitor.
* Optionally modify "InventoryReport" and "UpdateInventory" with a pathed location.
The example assumes the current directory.

Close and save the configuration file.

Click for larger view

Notes:  

The configuration file "FileChangeINI.MD5" is an INI file, but does not use an INI file-extension.  Reason:  Files with an ".MD5" extension are ignored by this utility and will not appear in their own inventory.  "MD5" files, can live in the same directory as the files you are inventorying.

If the sample INI file did not generate, you do not have enough rights in that directory.  Run from a DOS prompt to see the error message.  Likely issue:  See below, Advanced Installation 

Bulk TESTING:

Create a test "FileDirectory" (C:\TEMP) and populate the folder with expendable files.

1.  From a Command Prompt (DOS prompt), re-launch the program

C:
CD\Data\Monitoring\FileChangeMD5
FileChangeMD5.EXE  (it will use the default FileChangeINI.md5)

or
FileChangeMD5.exe  C:\<path>\filechangeINI.md5

Notes:
When launched, the EXE looks in the current directory for a config file, unless told otherwise.

Notes:
Optionally move the config/INI file to any directory, any name.
Launch with this command line: 
FileChangeMD5.exe C:\mypath\myconfigINI.MD5

Results:
This runs the program and builds a base-line inventory database of the (C:\Temp) directory.

2.  View the Results

Start, run "Notepad.exe"
Tunnel to the directory where the Program/INI was installed, or to the path specified in the INI.  File

Open this inventory file:  "Inventory.MD5"

* "Inventory.MD5" is the inventory of changed files and the first-time run is the base-run. 
Notice how the first-time run always shows all files as "New".

     Close Notepad.

C.  Run the executable a second time (with no file changes)

Using Notepad, re-open "Inventory.MD5"
All files now show as "unchanged" 
e.g. no changes to the base-inventory since the last run.

D.  Edit any or change any file in the inventory directory (C:\TEMP), making an innocuous change. Save the change.

Delete a different file
Add a new, unexpected file

E.  Run the executable a third time.

Using notepad, open file "InventoryReport.MD5" 
Note file changes in the status column:

   

To Force a re-inventory of all files in (C:\TEMP), setting a new base-line:

1.  Erase file "Inventory.MD5" and re-run the program.
     All files are re-detected as New.

To Force a single-file to be detected as "changed"
(for example, you need a downstream routine to re-process it), do the following:

1.  Using Notepad, open the current inventory file:  "Inventory.MD5"
 
2.  Edit the file's MD5 hash, changing to any random number.
     Changing only one digit is enough to invalidate the checksum

Click for larger view

3.  Re-run the program.  The file will show as a forced "changed"


Advanced Installation Notes:

Microsoft has restricted "C:\Program Files (x86)" and "C:\Program Files" from unsigned, and un-installed programs. 

If you are like me, you really want your utility programs in these folders, where they are protected.  The trouble is, the program wants to write a default configuration INI file in this same area.  Resolve with these steps:

Build a default FileChangeINI.MD5 control file:

A.  Temporarily copy "FileChangeMD5.exe" to a data folder (such as MyDownloads, C:\data, etc.)

B.  Launch the program one time (double-clicking).  It will have enough rights to write its default configuration file, "FileChangeINI.MD5" -- or copy the file from below and avoid the temporary steps.

C.  Move the executable, "FileChangeMD5.exe", back to your favorite ("C:\Program Files\Util", etc.) folder. 

D.  From the temporary copy's directory, move the newly-constructed INI file, "FileChangeINI.MD5" to any directory of your choosing, anywhere but C:\Program Files.

For example,  C:\Data\Jobs\FileChangeINI.MD5

E.  Edit the INI file, making two changes:

Set "InventoryReport" and "UpdateInventory" paths to a location where you have update rights; typically any location other than C:\Program Files.  For example, C:\Data\Monitoring

Set "FileDirectory" to the folder you want to monitor.

;FileChangeINI.MD5 Configuration File
FileDirectory     = C:\Temp             //Search this directory
FileMask          = *.*
LastRunInventory  = C:\Data\Monitoring\Inventory.MD5
InventoryReport   = C:\Data\Monitoring\InventoryReport.MD5
UpdateInventory   = TRUE

with this, the executable can live in C:\Program Files, and the inventory files can be directed to a folder where your normal Windows account has update rights>

F.  Launch with this command line, pointing to the INI file above:

C:\Program Files\Util\FileChangeMD5.exe  C:\Data\Jobs\FileChangeINI.MD5

The Executable can live in a protected directory.
The INI is in a known location, anywhere on the disk.  It points to an updateable area.
The Last-run Inventory file lives where you have rights T
he searched-File-Directory (C:\Temp) can be anywhere.

This solves the restrictive Windows security, without having to grant rights to odd places.

Command Line Options

This utility is best run from a DOS Command prompt and is designed to be run in a batch, automated, mode.  Except for the single-file report, the program will not pause or stop for user input.

A configuration file (FileChangeINI.MD5) controls how it behaves and where results are written.  All filenames can be pathed.  For example, the program can be configured to look at a server's UNC path, and store the inventory reports on a different drive, and the INI file could live on a third drive, as long as all are accessible by the user running the code.

Each time it runs, it overwrites previous inventories and reports.

Command line options:

FileChangeMD5.EXE  (no parameters)

Assumes the current directory contains the default-named
FileChangeINI.MD5 configuration file.

If FileChangeINI.MD5 does not exist, it will generate a default INI.
The INI will point to c:\Temp as a sample.
This assumes your Windows account has update rights in this folder.

FileChangeMD5.exe  C:\Path\FileChangeINI.MD5

Launches using a manually-specified configuration / INI file 
The INI file does not have-to-have an .MD5 extension, but is recommended.
If specified on the command line, the INI file can be any name, any extension.

FileChangeMD5.exe  C:\Path\FileChangeINI.MD5 FALSE

Runs in test, but does not make changes to the Inventory.

Possible Command Line Error:

When trying to run a single-file MD5 check, it is easy to forget the keyword "file=". 
The result is messy and the program will be confused, reporting "Warning: Unexpected value in prefs INI..."

Click for larger view

Bad Command line: 
FileChangeMD5.exe  C:\somefile.ext

Proper command line:
FileChangeMD5.exe  file="C:\somefile.ext"


I hope you enjoy using this program.  Drop me a note and tell me how you like it.

Version history:
1.00 - Initial release
1.01 - Added command-line /?  and /help logic
          Improved bad INI file detection with clearer error messages
          Improved missing file= parameter detection.
          Improved error message texts for Configuration/INI errors

2022.02 This program is way-overkill for a basic MD5 check.  I wrote a new Windows utility that is easier to use, but is not able to inventory an entire directory.

Link:  Midy5

Related keyliner Programs:
BullDozer -- a batch file delete program
Prize Select Raffle Program