Wednesday, June 28, 2017

File MD5 Checksum - How to tell if a file has changed

File MD5 Checksum - How to tell if a file has changed
File MD5 Report - Batch MD5 Reporting

A file's MD5 Checksum can tell you when a file has been modified from the author's original version.  For example, when downloading a program from the Internet, developers often will give you the file's MD5 check-sum -- your version's MD5 could be compared with the author's to see if there was a change.  

Seeing a file's MD5 checksum, sometimes called a 'hash,' requires software.  I have written a handy utility, which you can download. 

This utility can report the MD5-hash for a single file.  But more interestingly, it can keep track all MD5's in a directory -- keeping a mini-inventory.  With this, you can tell when a file has been added, changed or deleted! -- regardless of the filename or date-time-stamp.


Problem at the Office:

At the office, I had a directory of photographs that are weekly refreshed by another system.  Each week the entire inventory was replaced.  Original filenames and sizes remained the same, but because the files were re-downloaded each week, the date-time-stamp changed.  I needed a way to tell which files were different. 

This Keyliner utility resolved these problems.  It is able to tell if the same-named, same-sized photo contained different contents and it can tell who is new and who has been deleted. 


Download Utility

Download this Keyliner utility, from Keyliner's GDrive:

Link: FileChangeMD5.EXE

This utility is free for all personal and commercial use.
No registration.  No logging in.  No email.  No nag screens. 

Ironically, it has this MD5: 54-d5-e7-f6-59-7a-96-6b-6c-0e-f8-90-ac-95-b9-5d


Click the download link and save to a known directory -- but do not run from the browser (you won't be able to see the results).  The program is best run from a command-line prompt.  (Start, Run, CMD.  Then change to the download directory and type 'FileChangeMD5.exe'). 

Since the executable is not signed (too expensive), Windows will warn you of an un-trusted program downloaded from the internet.  If you trust my program, accept with a "Run anyway".  You will only be prompted for this once.





Features:
  • Show a specific file's MD5 check sum, using this command-line

    FileChangeMD5.exe file=filename.ext
     
  • Optionally, using a configuration file, report on all file-changes within a directory, showing
    -Changed Files
    -New Files
    -Deleted Files

    For example:
    FileChangeMD5.exe c:\temp\filechangeINI.MD5
     
  • Keeps track of changed files by using a text-based database - no database to install or maintain
  • Output is an ASCII tab-delimited file (default name: "Inventory.MD5"), suitable for Excel or other programming
     
  • Runs in Batch or interactive command-prompt mode
  • Runs unbelievably fast; directory is parsed in sub-seconds
     
  • Can report, grouping by changed-status, or by filename
  • Controlled with text-based INI files; multiple INI files, multiple directories, etc.
     
  • Free, no charge, no registration, no nags
  • Simple EXE - No installation required
  • No registry changes; no databases
     

Example Output:

This shows the inventory of all files in a directory (C:\Temp).  Notice the Status column:  "nochange", "changed", "new", and "deleted".  Each file's checksum/MD5 is listed:

Click for larger view

This file can be consumed by other software, parsing for the Status Column.  See also, the optional InventoryReport, below, for a similar inventory file.


Optional Inventory Reporting:

If the config file's "InventoryReport" field is populated with a file name, a secondary inventory report is generated.  The is a simplified report, showing only the type of change and the file name.






Individual File MD5 Checksum

The utility, without any configuration or setup, can report a single file's MD5 checksum, showing the results to the console. 

This example shows the MD5 hash for the file named "C:\Temp\file-01.txt"

Click for larger view

To view a single file, run the program from a Command prompt.
Use this command: 

filechangemd5.exe  file="C:\path\somefile.ext"

where the keyword "file=" is required. 
use quotes around the path and filename.

If you download this utility, use the downloaded program to check its own MD5!  Use this DOS command:

FileChangeMD5.exe fileChangeMD5.exe

Compare with my published hash near the top of this article.



Installation and Configuration:

There is no real installation to this program.  Just copy the EXE.  But if you want to inventory a directory, it requires minor notepad editing changes to a configuration file.

1.  Download FileChangeMD5.exe from Keyliner's GDrive

2.  Place executable file in any directory
     I recommend a data-folder, such as C:\Data\Programs\FileChangeMD5 or any directory.

Special Note:
If saved in "C:\Program Files"  or "C:\Program Files (x86)",
you would have to give your user-account unusual rights to update files in this protected area.
Putting the executable program in Data folder resolves this problem.

* See Advanced Installation, below, for more information on the Program Files directory.

3.  Build a sample configuration file.

     From a Command Prompt, or with Windows Explorer,
     Double-click (launch) the program for an initialization-run.

     This generates a default INI file, named "FileChangeINI.MD5
     The file arrives in the same directory as the executable. 
     (You must have rights to write the file in this directory)

Click for larger view


Notes:  
The configuration file "FileChangeINI.MD5" is an INI file, but does not use an INI file-extension.  Reason:  You do not want this file to show up in the inventory database.  Files with an ".MD5" extension are ignored by the utility.  With this, the files "Inventory.MD5" and "FileChangeINI.MD5", can live in the same directory as the files you are inventorying.

If the INI file is not generated, you do not have enough rights in that directory.  Run from a DOS prompt to see the error message.  Likely issue:  See below, Advanced Installation 


4.  Using Notepad, edit the file and change the FileDirectory path from
     C:\Temp  to a directory of your choosing.

     Close and save the configuration file.



TESTING:

Create a test directory (C:\TEMP) and populate the folder with expendable files.

A.  From a Command Prompt (DOS prompt), re-launch the program

     C:
  CD\Data\Programs\FileChangeMD5
  FileChangeMD5.EXE


     This runs the program and builds an inventory of the (C:\Temp) directory.
     The exe must be in a folder where your user-account has rights to update files.
     Often, Program Files folders are restricted.  See below for more advanced options.


Notes: 
When launched, the EXE looks in the current directory for a config file, unless told otherwise.

Notes:
Optionally move the config/INI file to any directory, any name.
Launch with this command line: 
FileChangeMD5.exe C:\mypath\myconfigINI.MD5


B.  Start, run "Notepad.exe"

      Tunnel to the directory where the Program/INI was installed.
      Select file-open, and open this file: "Inventory.MD5"

  * "Inventory.MD5" is the inventory of changed files and the first-time run
     is the base-run.  

     The first-time run, all files show as "New". 



     Close Notepad.

C.  Run the executable a second time (with no file changes)

      Using Notepad, re-open "Inventory.MD5"
      All files now show as "unchanged"  e.g. no changes to the base-inventory since the last run.



D.  Edit any test file in (C:\TEMP), making any innocuous change. Save the change.

     Consider deleting a file
     Consider adding a new, unexpected file

E.  After changing files, run the executable a third time.

     Using notepad, open file "InventoryReport.MD5"
     Note file changes in the status column:

   

To Force a re-inventory of all files in (C:\TEMP)

1.  Erase file "Inventory.MD5" and re-run the program.
     All files are re-detected as New.

To Force a single-file to be detected as "changed"
(for example, you need a downstream routine to re-process it), do the following:

1.  Using Notepad, open the current inventory file:  "Inventory.MD5"
 
2.  Edit the file's MD5 hash, changing to any random number.
     You only need to change one digit to invalidate the checksum

Click for larger view
3.  Re-run the program.  The file will show as "changed"


Advanced Installation Notes:

Microsoft has restricted "C:\Program Files (x86)" and "C:\Program Files" from unsigned, and un-installed programs.  If you are like me, you really want your utility programs in these folders, where they are protected.  The trouble is, the program wants to write a default configuration INI file in this same area.  This is resolved with these steps:

Build a default FileChangeINI.MD5 control file:

A.  Temporarily copy "FileChangeMD5.exe" to a data folder (such as MyDownloads, C:\data, etc.)

B.  Launch the program one time (double-clicking).  It will have enough rights to write its default configuration file, "FileChangeINI.MD5" -- or copy the file from below and avoid the temporary steps.

C.  Move the executable, "FileChangeMD5.exe", back to your favorite ("C:\Program Files\Util", etc.) folder.  The temporary copy in (MyDownloads, c:\data, etc.) can be deleted.

D.  From the temporary copy's directory, move the newly-constructed INI file, "FileChangeINI.MD5" to any directory of your choosing.

      For example,  C:\Data\Jobs\FileChangeINI.MD5

E.  Edit the INI file, and set a path for the Inventory files to a location where you have update rights.  Typically any location other than C:\Program Files.  For example, "C:\data\jobs\Inventory.md5":

;FileChangeINI.MD5 Configuration File
FileDirectory     = C:\Temp             //Search this directory
FileMask          = *.*
LastRunInventory  = C:\Data\Jobs\Inventory.MD5
InventoryReport   = C:\Data\Jobs\InventoryReport.MD5
UpdateInventory   = TRUE


with this, the executable can live in C:\Program Files, and the inventory files can be directed to a folder where your normal Windows account has update rights -- such as "C:\Data\Jobs\Inventory.MD5"

F.  Launch with this command line:

C:\Program Files\Util\FileChangeMD5.exe  C:\Data\Jobs\FileChangeINI.MD5

The Executable can live in a protected directory.
The INI is in a known location, anywhere on the disk.  It points to an updateable area.
The Last-run Inventory file lives where you have rights
The searched-File-Directory (C:\Temp) can be anywhere.

This solves the restrictive Windows security, without having to grant rights to odd places.



Command Line Options

This utility is best run from a DOS Command prompt and is designed to be run in a batch, automated, mode.  Except for the single-file report, the program will not pause or stop for user input.

A configuration file (FileChangeINI.MD5) controls how it behaves and where results are written.  All filenames can be pathed.  For example, the program can be configured to look at a server's UNC path, and store the inventory reports on a different drive, and the INI file could live on a third drive, as long as all are accessible by the user running the code.

Each time it runs, it overwrites previous inventories and reports.

Command line options:

FileChangeMD5.EXE  (no parameters)

     Assumes the current directory contains the default-named
     FileChangeINI.MD5 configuration file.

    If FileChangeINI.MD5 does not exist, it will generate a default INI.
    The INI will point to c:\Temp as a sample.

    This assumes your Windows account has update rights in this folder.


FileChangeMD5.exe  C:\Path\FileChangeINI.MD5

    Launches using a manually-specified configuration / INI file
    The INI file does not have-to-have an .MD5 extension, but is recommended.
    If specified on the command line, the INI file can be any name, any extension.


FileChangeMD5.exe  C:\Path\FileChangeINI.MD5 FALSE

     Runs in test, but does not make changes to the Inventory.


Possible Command Line Error:

When trying to run a single-file MD5 check, it is easy to forget the keyword "file=".  The result is messy and the program will be confused, reporting "Warning: Unexpected value in prefs INI..."

Click for larger view
Bad Command line: 
FileChangeMD5.exe  C:\somefile.ext

Proper command line:
FileChangeMD5.exe  file="C:\somefile.ext"




I hope you enjoy using this program.  Drop me a note and tell me how you like it.

Version history:
1.00 - Initial release
1.01 - Added command-line /?  and /help logic
          Improved bad INI file detection with clearer error messages
          Improved missing file= parameter detection.
          Improved error message texts for Configuration/INI errors


Of Interest:
A one pixel change in a photograph, changing a white pixel to light grey, resulted in this MD5 change:

MD5: 9f-46-58-ef-3d-fe-76-45-65-61-f0-d3-a7-f3-62-bc     Original File; saved twice
MD5: 54-d3-9d-69-07-1a-9f-d9-92-1c-1e-2b-54-42-de-a4   One-pixel change


Related Keyliner Programs:
BullDozer -- a batch file delete program
Prize Select Raffle Program

Saturday, June 24, 2017

WordPerfect Compliment

If you saw my previous post about self-publishing my C# programming manual on Amazon.com's Kindle, then you would know it was a large project.


The book, now divided into 3 volumes, 28 chapters, has passed 1,800 pages.  The last time I counted, it had nearly 1,000 illustrations.

A serious document that no traditional publisher would touch.  Plus, it was a crowded market with lots of similar titles.  But I know my book is one of the better ones out there.... 

After all this work, I still want the programming manual out in the real world.  Because of this, I chose to publish each chapter as a separate Kindle title, charging $2 or $3 -- making for a low-cost of entry. 

Editing 28 individual documents would have been a chore -- I needed to work with them as one large, multi-chapter book because page-numbering, TOC's and Indexes needed to span chapters.

How did I manage?  WordPerfect!


WordPerfect?

Most of you just snorted. "Tim still uses WordPerfect?  Sheese!" 

Microsoft Word would die on a document like this -- and I would still be cursing squirrelly graphics and bullet-lists.  I stand by my guns here.  WP knows how to handle complicated documents.

Case-in-point, today I was editing Volume 1, an 800-page documents behemoth (a master, with subdocuments, all expanded and all visible). 

Scrolling is effortless, moving from top-to-bottom and anywhere in-between, without delays.All edits and graphic changes are perfectly tracked.  Font and margin changes are made in the master document - one change and all chapters change. 

A click of the mouse re-generates the Table-of-Contents and the index rebuilds with a concordance file; this takes about 5 seconds.  Let me remind you, this is on an 800+ page document, with hundreds of tables, margin changes, and the like -- and all the graphics are right where they are supposed to be, flowing with the text without a problem.

When I save, each chapter parks itself out to the disk as a separate file and I don't have to keep track of them.  If I want, I can edit the chapters individually -- but seldom do.

WP to PDF to Kindle - Easy Cheesy

From the master document, I printed each chapter using WordPerfect's PDF option, and fed the results into the Kindle Text Book Creator.  The result - a perfect document, exactly as formatted.  Uploaded to Amazon, clicked Publish, and was done.  Super easy and kind-of-fun.

Even if I were using Word, I would still print to PDF and then import into Kindle -- this saves the formatting and graphic positions.  (But if I were using Word, the documents would not be as well-behaved.  Yes, I am opinionated about this.)

What is my point?  Don't be a snob and say Microsoft Word is the only word processor for you.  WordPerfect Student/Home is $50 -- and it saves in DOCX format.  If you are frustrated with your current word processor, try someone else.


Related Keyliner Articles:
Publishing Text books and Comic books to Kindle

WordPerfect Reviews  X5 - X8
WordPerfect for School Papers
WordPerfect Protecting Text across page boundaries

Shameless Amazon link:  A Beginner's Guide to C-Sharp Programming
https://www.amazon.com/s/ref=nb_sb_noss_2/134-2760193-8662735?url=search-alias%3Daps&field-keywords=ABGC



Monday, June 19, 2017

Kindle Book Publishing

I just published my C-Sharp programming manual on Amazon's Kindle.  The process was easier than expected.

First, some history as I understand it:  On Kindle (or Nook), authors can upload MS Word documents and the software will convert it to the e-book reader's format.  Page numbering, chapters, etc., are all figured out during the upload.  The end result is a document that changes the number of pages, depending on the size of the device and on the magnification of the font.  Things like indenting, columns, and other such formatting are ignored.

I admit I have not actually done this because I knew my book -- a text book -- would die during the conversion.

Technical and illustrated books do not tolerate a simple text conversions.  Page-numbering, page-breaks, and especially illustrations come into play, and the e-book cannot re-format the document without causing harm to its structure and design. For the same reasons, children's book and comic books also require special care.

Enter Kindle's new (beta) software -- "Kindle Text Book Creator."

This tool takes your already-formatted, book-ready-PDF, and makes it an e-book.  Pagination and illustrations survive and the resulting e-book is exactly as the PDF.  Download and install to your workstation as a local application.  There is no cost.



Steps:

1.  Download and install the textbook creator program.
2.  Launch.  File-New, Open the PDF.  Then choose "Package."  The result is a "kpf"  (kindle package file).

3.  Login to Amazon.com's Kindle Direct Publishing Site: https://kdp.amazon.com

4.  Click "Create a new Title "+Kindle eBook" - not Paperback.

5.  Upload the kpf and follow other on-screen prompts.

Within 24hrs or so, the book is available in their catalog, ready to sell.



A Beginner's Guide to C-Sharp - A la'carte

My book was so obscenely big*, no traditional publisher accepted it.  Plus, the market is saturated with similar titles.  For these reasons, I thought I'd try Kindle and self-publish.

*how big, you ask?  3 volumes, 28-chapters, 1,800 pages, 1,000 illustrations.  Untold code-block examples.

As an experiment, I decided to be different.  Each chapter is published as a separate "book" and each chapter was priced at $2 or $3 dollars, depending.  This way, students could pick and choose which chapters to download and would not be forced to buy three volumes all at once.

This design is causing me troubles.  The books are hard to differentiate from each other when browsing them on a tablet.  The price is not displayed on tablets or smaller devices, obliviating the benefit of my chapter-by-chapter design. From the web, it flows a little better.

Success?

Success will be judged if anyone buys the book.  With Amazon taking a 70% cut, there is no real profit here.  An average chapter earns less than a dollar.  Having only done this work today, it is too early to tell.  I will come back to this article and update my status.

If you are curious, search on Amazon for "ABGC" (A Beginner's Guide to C-Sharp).  Ignore all those other similar publications -- mine is clearly better.


Lessons Learned:

I have since revisited the uploaded chapters and made three changes to them: 

First, each chapter got its own graphic.  Although all 28 chapters are from the same book, each chapter needed its own graphic.  On a phone or tablet, each chapter's title was truncated and since the graphics were all the same, it was hard to tell which chapter-was-which. (If I published the book as one giant document, this would not have been a problem).


Secondly, each chapter began with a Table of Contents for the entire volume - 8 pages.  When Kindle readers downloaded a sample, they would only get the cover page and the TOC -- and could not see the document.  I removed the TOC from all of the intermediate chapters, leaving it only on the first chapter of each Volume.  This was not a perfect solution, but not bad.

Finally, the low-cost per chapter is again hidden on the smaller devices and the low price is not discoverable until too late.  In the first page of each document, on the first paragraph, I mention the low-price per chapter.  Later, I will update the introductory text and make this more prominent.  Unfortunately, this makes it difficult to change the prices -- sales and price increases will now require multiple edits per document.

Tuesday, June 13, 2017

Greenshot Screen Capture - The best

Utility: Greenshot Screen Capture

When I need screenshot illustrations for this blog or for documentation at the office, I use a utility called Greenshot.

http://getgreenshot.org/downloads/

Greenshot is a free, no advertisement, no registration, no-strings-attached utility.  The authors ask for a donation; they do not nag.  I recommend this program


How it works:

After installing, note the "Greenshot" icon in the "G" system-tray, meaning the program is running and waiting.

At any time, press the keyboard's Print Screen key and draw a box around the part of the screen to capture.  (Other keystrokes will capture the entire screen, the same area as last time, etc., as well as other options to boring to list here.) 

For example, her is a screenshot of Windows Explorer.  Notice the size, in pixels.  The program can optionally capture the mouse pointer and it can show a pixel magnifier. 


Then it prompts for a location:


I typically choose "copy to clipboard" or "Open in Image Editor".

"Open in Image Editor" is my favorite destination -- This option is slightly misnamed.  By default, it opens in a small Greenshot editor -- not in your photo editor.  From here, you can manually copy to the the clipboard, where I then paste into my favorite graphic editor.


I like the "Open in Image Editor" because it opens a new window with each Print Screen, acting like multiple clipboards.  I might have a dozen of these open, then, as needed, I copy the image to the Windows clipboard and manually paste into my graphics editing programs.  This helps keep my work organized.

You can also write automatically to a numbered file-folder; a feature I have not used, but understand its usefulness.

The Image Editor even has click-to-place counters!  Oh, my heart, be still!  As well as line-drawing and shape tools....



Preferences:

Because of new security with Windows 8 and newer, you may find the PrintScreen key does not work on all screens you are trying to capture (pressing PrintScreen does not activate the highlight).  I made the following changes:

A.  From the System Tray, select the Greenshot icon, Preferences.  In the General tab, set the program to not [  ] Launch Greenshot on startup.

B.  Then, from the Start Menu, type the letters, "Greenshot".  When the program is found, other-mouse-click the icon and "Pin to Start"

C.  Launch the program manually by "other-mouse-clicking" the Greenshot desktop tile, choose "More" and then "Run as Administrator"

With this, you will have full control, on any screen.  Greenshot's FAQ explains the reasoning behind this.


I continued with these other preferences, set one-time: 

From the System Tray icon, set preferences. 

a.  I like to change the default Output Destinations to "Open in Image Editor":  This gives me multiple clipboard like functionality.


Cautions:

Download the utility from the Vendor's site (link at the top of this article); do not download from SourceForge or other sites because many have re-packaged the program around banner ads, browser changes and other nefarious stuff.  I learned this from personal experience.

Other comments:

The source code is available for the curious and there is a good FAQ section on the website, which will answer any other questions you might have.


Related Links:
http://getgreenshot.org/downloads/

My favorite graphics/photo editor:  Corel Paintshop Pro - used for all illustrations here.


Friday, June 9, 2017

Computer Household Cleaning

Computer Household Cleaning

Keyliner has written several articles on steps to cleanup viruses and other Windows infestations.  In particular, I use this this Keyliner article to recover other people's machines:  Virus Cleanup Steps.

But if your machine is running reasonably well, this series of articles, in particular, "Windows Repair and Maintenance" / "Cleaning and optimizing a Windows computer" was interesting.  I recommend reading the articles in this site: 

https://decentsecurity.com/

Use my article to recover a machine that is horribly infested, then follow-up with DescentSecurity's article. I don't agree with everything said, but most of the suggestions are good.  Here are some areas I found particularly interesting and I have been or am now doing:

1.  Run Windows 10.

Windows 7 is too old and Microsoft's resources are dedicated to Windows 10.  The newer OS is more secure and safer. 


2.  Set Windows UAC Nags to maximum.

By default, Windows sets this one notch lower and that is where I used to be set; I have now changed.  Since I am not installing a lot of new software, the change will not add to my burdens. 

To check the settings:
From the Start Menu, type "UAC"

Some people turn UAC off.  They are idiots.


3.  In Google Chrome and in Firefox, install "uBlock Origin" (Addon / Plugin)

uBlock instead of AdBlock plus (my long-time favorite plugin)
uBlock does more to stop drive-by virus installs
It has a larger library to search than Adblock
It is faster than all other adblocking software and does more stuff


4.  Uninstall Java. 

I have never considered this, but in retrospect, agree.  According to the author, "Java is not needed for any modern reason other than obscure software".  Note this is different than JavaScript (used by most websites).  Java can be uninstalled; it will not affect JavaScript.
 
See Programs and Features, un-install Sun Java.

Related: The article suggests upgrading Flash.  On my main Windows 10 machine, I have it disabled.  This has caused problems, but I remain in that state.  If curious, see this keyliner article: Time to Disable Flash.


5.  WinSxs Cleanup

I have long known the WinSXS directory is a pig sty of duplicate DLL's.  DescentSecurity recommends running this Microsoft command, from a DOS prompt, run as Administrator:

C:\WINDOWS\system32\dism.exe /online /cleanup-image /startcomponentcleanup /resetbase

An intriguing idea.  However, on my machine, it ran without errors, but had no affect on my disk.  The number of files and directories in WinSxS remained the same.  Your results may vary.


6.  Registry Cleaners and third-party Windows Optimizer programs are useless

These are snake oil and often dangerous.  I have long said this.  Yes, there are times to fiddle in the registry, but use care.  Registry Cleaners do not.


Microsoft Wish*

I wish Microsoft would allow us to customize the UAC nag screen.  Much like some banking sites, if the UAC showed a personal photograph or some other personalization, we would know we were on a non-spoofed screen.  Even better, imagine your virus scanner with such a personalization.



At the same time, the picture would be our family and employee's cue to never click "OK".  Tell them to call the IT Support Staff (you), before clicking OK.  This would help stop drive-by installs (such as, "You must install this driver to see this online video!"  -- almost guaranteed to be a virus.