Web Fingerprinting

 Web Fingerprinting

Sites can use Java Script to query your workstation and make a "fingerprint" of your hardware and software uniquely identifying you.  In the old days, this was done by storing a third-party cookie.  Now newer techniques are more sophisticated and are stored outside of your control. 

This harms in untold ways.  For example, while searching for airline tickets, if you bounce between airlines, looking for the best fare, you will often see the rate will creep upwards each time you return to review the price.  The hope is to scare you into committing to the purchase before it rises higher.  

They are able to do this because they can tell you are the same computer returning to the same site.

This is fun to review, but it is unclear
the actions you can take to prevent

The tracking is tied to many things, including your external-facing IP address (which is hard to change), the cookies stored on your computer, the browser version being used, the video card, and the number of fonts installed.  This is called a "fingerprint."  This tags your machine, not you as a user-id.

Am I Unique

Click this link and let them display your workstation's fingerprint.  "AmIUnique" and "EFF.org's" sites will show your digital "fingerprint" and it is beyond interesting. 

https://amiunique.org

See also this site (same idea, more educational):
https://coveryourtracks.eff.org/learn

The gist is this:  Yes, you are probably unique and can be easily identified.  This happens even if you are using a boring PC with a normal video card, and a normal operating system.  Click the link to see how identifiable your PC is.

For example, my first attempt at AmIUnique said, yes, I am absolutely unique to all of the people who have visited this site in the past (compared with the 3 million current visitors in their database):

The test shows 20 or so attributes.  Some are marked Green (which means you are not differentiated from other users -- you blend into the crowd) and some are marked Yellow or Red, which means this can be used to target your machine.  The more Red you have, the more unique you are.

Click for larger view

User Agent Tag

The "User Agent" tag is particularly interesting.  This is what your browser broadcasts to the world and it is transmitted in a "header."  For years, browsers have lied about this -- telling websites they are a more-or-less generic browser.  Amazingly, almost all browsers (including Chrome, IE, and others) claim to be Mozilla. 

Everyone should look the same but sadly they also return a version so sites can detect old browsers and display the page differently to accommodate them.  This helps identify your computer.

At first, I thought I was using Mozilla Firefox and it shows as "Mozilla" (just because almost all browsers claim this), but at the tail-end was a version number.  It reports, "I am unusual, a bit odd, and identifiable."  This awards me a flunk in "Am I Unique's" calculations:

My friend checked on his iPhone (Safari), and from his Surface Pro (MS Edge), and it also shows a similarly bad score.  In other words, his browser, which was boring-as-hell, still gave a red score in this category, and when all was said and done, he was unique compared the other 3 million people.  I am a bit confused about how this can be.

Changing the UserAgent

Your machine's total fingerprint is a collection of discoveries, and the User Agent is one of the most important.  Because I use Firefox, it targets me as a particularly odd duck.  But keep in mind, Microsoft Edge, and Chrome also have the same issue.
 
This is not recommended, but you can override what the browser sends by manually setting the User-Agent to some other, more generic value.  With this, you can try to blend into the crowd.  Indeed, this worked at a superficial level and my score improved.  As you will see, this caused problems in later testing.

Change the UserAgent/Header 

In Firefox:
a.  Browse to this url:  "about:config"  (no quotes)
b.  Accept the warning message

c.  Type:  "general.useragent.override"  (no quotes, case-sensitive!)

d.  Select (*) String, click "+"
e.  Paste this value, where I am pasting Microsoft Edge's string,
     Click the checkmark.
     Close the window

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.2903.86

In Chrome/Edge
a.  From a browser page, press F12 (opening Developer Tools)
b.  Press Ctrl-Shift-P  (opening a command menu)
c.  type "network conditions"  (now quotes)

d.  In the bottom section, scroll down, locating "User Agent"
e.  uncheck [ ] Use Browser Default

f.  Paste the value from above (see Firefox)
g. Close the developer tools box ("x"), upper-right

Results:
This changed my UserAgent flag from a red 0.27% (which means highly identifiable) to a yellow 16.22% (which means in a crowded field).

Note: This still has a version number.  If you remove the version number, you will become even more unique because nobody will have done that.  Even if you manually set a new (and different / setting an older, more common version number), after a while, after everyone upgrades to newer versions of Chrome, this will slowly devolve into being more unique -- think more odd.  If you go this route, periodically, reset this to a new base line. 

Chrome, realizing this is a tracking problem, has promised to start removing versioning from this string.  They have not completed this yet (2024.12). See this article: 

https://developers.google.com/privacy-sandbox/blog/user-agent-reduction-android-model-and-version

where the latest EDGE browser (2024.12) shows:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.2903.86

The latest Chrome browser shows (2024.12) shows:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36

Find the current User Agent tags here:
https://explore.whatismybrowser.com/useragents/explore

In the future, once (Chrome, and others) finally get around to removing the version number from this string, revert your manual changes back to the automatic broadcast.  Undo this change by (firefox) returning to this config value and delete the newly-added key.

But there is a problem: 

Although changing my UserAgent improved the browser's "hiding-in-the-crowd-ness", it prevented me from logging into GMail or into Google's blogging tool (this article).  Google said, "this is an unsupported browser," even though millions of other browsers use the same string.  This means something else is leaking -- something else is telling the website that there is a mismatch between the "header" and the browser itself. 


Trouble on the Horizon

The next area is the fingerprint is a list of fonts.  With JavaScript, websites can query the fonts on your system.  For example, my system has 180 fonts and is flagged as highly unique.  Your font list is also probably equally unique. 

I don't know how to defend against this.  It seems no matter what I could do here; my list would still be unique.  The only good solution would be a setting in the browser that says, "Return a simplified, standard list of fonts" -- but all browsers would have to agree to do this, and this is unlikely.

Perhaps another solution would be to periodically add or remove inconsequential fonts from your operating system's font list, just to add variation.  This would break the tie to your fingerprint, causing another fingerprint to be generated.  Naturally, this makes your list even more specialized.  I'd pick an obscure font and remove the Bold, then later the Italic, then later re-add the font back.  This is a lot of monkey work.  Perhaps I might write a program to automate this (check back in the future).

More Trouble

In the AmIUnique site, there are two references to video/image settings, and the website can tunnel into the hardware and tag your machine's video card.  This article describes the technique:  

https://blog.amiunique.org/an-explicative-article-on-drawnapart-a-gpu-fingerprinting-technique

It queries your exact card by uploading a secret image and running a test to detect minor variations in CPU and processing speeds.  With this, it is able to identify your video card uniquely -- even among all the other identical video cards sold from the same company. 

The article discusses mitigations, none are palatable and this too seems hard to defend against.  On the plus side, this is just one of several parameters that need to align in order to detect your machine.


JavaScript

Of course, all of this nonsense can be stopped by disabling JavaScript on a site-by-site basis.  Firefox has a neat plugin called, "Disable JavaScript" (see this link:  https://addons.mozilla.org/en-US/firefox/addon/disable-javascript).

This plugin immediately stops fingerprinting.  And it is also (sometimes!) useful for bypassing paywalls on news-reading sites.*  But it does not work well in sites, such as Amazon, or your airline, or this blogging tool -- those sites require JavaScript for basic functionality and whine when JavaScript is gone.   

I use this plugin.  On a site-by-site basis, I'll disable or enable JavaScript.  It works fairly-but-inconsistently well.  The plugin keeps track of the sites where you use this, making it transparent, once set.  The trouble is, it does not work on all sites.  But this is a low-risk, low-effort to try.

* Journalists deserve to be paid for their work, but the current design, where you have to subscribe to each news site is unmanageable.  When you randomly and infrequently visit a publisher from a news aggregator service, such as Google News, the subscription idea doesn't work very well.  

Returning to Cookies

Sites can store fingerprinting as a cookie on your local workstation (but most now store them on their own databases, where there is not visibility).

Regardless, it is still wise to manage your third-party cookies and all browsers have controls in this area.  For example, Firefox, in "Settings,", "Privacy and Security", you can block cross-site and third-party cookies:

Chrome/Edge has similar settings, not illustrated.

2024.12 - Plus, this week, Firefox announced they are removing the "Do Not Track" feature -- why?  Neither reputable or disreputable sites paid attention to it.

Painfully, you could periodically clear all cookies (doing so from each browser you use).  This resets default file folder settings, upsets previously-typed user-id's and passwords, and other such nonsense.  When cleared, it is vaguely annoying.  I used to clear all cookies frequently -- but do so less often now, just because the fingerprints are usually stored on that site's servers. 

Clearing cookies only helps if the website stores the fingerprint in a cookie; your workstation would re-generate the same fingerprint, regardless.  Of course, cookies can store other information too.  Cookies are not inherently bad, but they can be used by third-parties for cross-site data.  If you ever wondered why you are seeing a ton of ads for wrist watches, when you one-time clicked a wrist-watch ad, blame your cookies.

However, it is 'sometimes' helpful to clear all cookies prior to price-matching (airlines, hotels.com, and other such sites).  Do this on your initial foray into the price comparisons, then a second time when you are ready to commit to the purchase.  Or better, use one computer for the initial research and a second to complete the purchase.  If that second computer were on a separate IP-address-range (such as a cellular hotspot), that would be even better.  

Brave Browser

Finally, consider a different browser, at least for some surfing activities.  
https://brave.com

This browser automatically passes you through a VPN, fixing the IP Address problem.  
It blocks fingerprinting (by presenting the same fingerprint for all users)
It blocks 3rd party and cross-site cookies
Blocks most ads (except their own)
Has the ability to make BAT (micropayments) to publishers

But they have bills to pay, and they present ads on their own landing page, and the ads often target Crypto.  ZDNet had comments in this article:  https://www.zdnet.com/article/brave-browser-the-bad-and-the-ugly  and this wikipedia article talks about other advertising-related issues with the Brave browser:  https://en.wikipedia.org/wiki/Brave_(web_browser)#Controversies

I've not yet tried Brave, but am considering it for some browsing tasks (competitive purchases, airbnb, airlines, etc.).

Final Notes: 
You may have noticed this article is a little scatter-brained.  I'm still pondering what is happening here.  Regardless, I found it interesting.  When I reviewed my friend's iphone and Surface laptop's report, I came to the conclusion that all machines are uniquely fingerprinted.  In other words, they know when you've returned.

I'm considering changing my asset-tag/management program:  DeviceID, making it periodically change fonts and a few other settings, in an effort to make my calculated fingerprint less predictable.  This article is my first thoughts on the matter.
 

Your comments welcome.

Related links: 
A keyliner article:  Network wide blocking of advertising.  I do this and highly recommend:
https://keyliner.blogspot.com/2018/01/network-wide-blocking-of-ads-tracking.html

keyliner.com's free asset-tag/management program:  DeviceID

DirectoryPulse - Keyliner Backup Software

DirectoryPulse - keyliner Backup Software

keyliner has written a free Windows backup program.
-2024.08 - Version 2.00 - Updated with new features

Backup large swaths of directories to a separate backup drive.  Files can be saved in dated folders, as compressed files, or as generational backups (multiple versions of files).  Jobs can be saved and re-loaded, and scheduled for automatic runs. 

Backup are non-proprietary and fast!

Write to local drives, SAN drives, and cloud drives, organized by project, by date, etc.

(Click for larger view)

Features:

* Backup to a local disk, SAN, One-drive, GDrive, etc.
* Non-proprietary backup - straight-forward file-copies

* Backup to optional dated-folders (YYYY-MMDD)
* Supports Generational backups
   (multiple versions of same file, V1, V2, etc.)

* Optional ZIP backups
   (reduce upload times to OneDrive)

* Can skip non-changed files
   Automatically skip cache folders, temp files, and hibernation files
   Automatically skips Windows System Files
   Skips One-drive virtual file links (saving re-downloading)

* Manages its own backup inventory, erasing old backups
* Configurations can be saved for easy retrieve and re-run
* Automate and schedule with Windows Scheduler
* Occupies zero RAM or resources when not in use; no background tasks

* Includes a nifty Directory Report that displays a sortable report, showing file-sizes and counts in a way that file explorer does not.  Use this to find your biggest disk hogs.

DirectoryPulse was originally written as a Student and Instructor exercise in Volume 6 of the Computer Programming Book, War and Peace Programming in C#  -- Amazon Kindle, written and published by keyliner.  It grew into this program.

Try it:

See below for download and installation instructions.

1.  Launch program, "DirectoryPulse.exe" from the Start Menu tile or by double-clicking the .exe.

The program was written with C#, .DotNet 8.0. When first launched, you may be prompted to install Microsoft's .DotNet 8.0 libraries.

2.  On first-launch, you are prompted for default backup destinations.  These are one-time settings. 

Most users setup these destinations:

D:\drive (local secondary drive, or USB disk)
SAN  (local network disk, or assign to a USB disk)
Cloud  (Onedrive or GDrive backup folder)

For each destination, type or browse to a top-level "backup" folder name -- such as "Backups"; all backups, of any type, will live in this folder.  (Optionally, browse to each destination.  Use the word "none" to disable the SAN or CLOUD destinations.) 

Click for larger view

Save the changes and return to the main panel. 
The program builds the top-level destination directory automatically when you close the panel. 

3.  On the main panel, select which directories to backup.    

In the top-most field, type the path, browse, or use File Explorer to drag-n-drop a folder onto the target. 
(DirectoryPulse! only cares about folder-backups; you cannot select an individual file.)

Optionally, right-mouse-click the field and choose "Add User Profile Documents".
+Multiple folders can be stacked into the backup job.

When done adding folders,

4.  Click Button "Refresh DOS Directory List"

This inventories the folders and shows a nifty report, showing all directories and subdirectories, with file counts and sizes. Sort by column-headings; right-mouse-click for a context menu.  This list is used by the backup.

5.  Session Settings: 

Set a backup destination and other preferences by clicking the Gear icon:

Click the "Gear" icon (-- this job's configuration settings)
Name this configuration file (e.g. "MyFavoriteBackup" or "Local-DataFolder")
Type a short description
Choose the type of backup ("Dated Folder [1-10]" - recommended)

Click for larger view

Set a pre-named sub-folder (which helps to build-out a destination-path).  Subfolders, such as "Projects", "Monthly", "Adhoc" are offered. 

Choose Adhoc or (None) if you have no opinion.  I typically use "Projects" (aka all my project backups go to the Project folder)

Once the subfolder is selected, a suggested backup destination path is constructed.  This is an assembled path, but can be changed manually to any other path (destination).  Destination folders are built automatically when the backup starts. 

The default subfolder names and other paths are adjustable.  See "System Defaults", this panel.

See the next article for Detailed Switch documentation.

6.  "Save" these preferences or click "Apply" to only use for this session.

7.  From the main screen, click bottom-row button "Backup"

The backup runs.  The backup re-creates the source-path (backup folders) in the destination path, creating each directory, one-for-one.  For example, if backing-up C:\data\subfolder1, the backup destination might be D:\Backups\adhoc\data\subfolder1.

When done, a report displays, along with a log file.

You are done!

If the backup type is "Dated Folder [1-10]", the backup destination has a subfolder: 
For example:  D:\Backups\Adhoc\2024.0804.  In this folder is a duplicate of the source drive's directory structure, with all eligible files.

Each time this configuration is run, it creates a new folder, keeping 10 generations of backups.  On the 11th backup (the 11th date), the oldest date is automatically deleted -- a self-cleaning backup.

Details and additional program documentation can be found here:
Detailed Switch documentation

Installation Steps:

DirectoryPulse is free to download and use for personal and commercial use. 
No registration, no login, no email. No advertisements, no nags, no spying.
Keyliner does not (and cannot) track who downloads or runs this program.

Installation is easy:
Download the .exe (.zip) and copy its supporting files into any folder on your hard disk.
Double-click the .exe to run - no installation required. 

Using the .exe from a download folder, or copying to a (my Documents) folder is a quick workaround for various Windows security concerns.  Some vendors recommend this, but these folders are inappropriate for executable software.  Instead, the program should be copied to Program Files so it gains the protection of other Windows security features. Total time: about a minute.

Follow these steps for a more professional installation:

A.  Download the .exe and support files (as a .zip) to a Download or Temp folder:

From Keyliner's public GDrive, click this link and download to a local temp or download directory.  

Version 2.02 Download Link
https://drive.google.com/file/d/1G-1eMXWpVW8UF5XFat8yk4oib2gZr-3A/view?usp=drive_link

When downloading, different browsers behave differently.
Select "Save-As"

Windows security will not let you download directly into Program Files -- you must save to a temp directory (technically, you will not be able to remove the "mark of the web" if downloaded directly into Program Files).

Since keyliner cannot afford a signing certificate, you will be prompted that the file is not safe (being downloaded from the internet).  Click "more information" and allow the program to download/run.

Zip File Hashes:
MD5: b3-83-aa-49-54-2e-5b-23-a6-6e-07-25-61-2a-1f-d1
SHA256: 7cf33b0aa486c6979cdb1825163fae12a682cf00882183e456787fddeeaf5f22

B.  Mark the download as safe-to-run:

Using File Explorer,

Right-mouse-click the downloaded DirectoryPulse.zip
Select "Properties"
Check [x] Unblock.  (This removes the "mark of the web.")

                 Click for larger view

* Only do this if you trust keyliner *and* only if downloaded from keyliner's public GDrive. 

If "Unblock" is not visible, it has already been unlocked (by Microsoft Edge).
Once [x] Unblocked is clicked, this security menu disappears.

C.  Create a Program folder to hold the program:

Using File Explorer, open folder C:\Program Files,
Create a new utility folder:  C:\Program Files\Util

D.  Copy the .exe and two support files to ProgramFiles\Util:

Using File Explorer,
Double-click to open the .zip folder

Copy/paste three files from the zip,
Paste to C:\Program Files\Util

DirectoryPulse.exe
DirectoryPulse.dll
DirectoryPulse.runtimeconfig.json

Do this copy as a two-step, copying from the temp/download folder, then into Program Files.  

E.  Create a Start Menu Tile:

Using File Explorer,
Tunnel to C:\Program Files\Util
Right-mouse-click the DirectoryPulse.exe and "Pin to Start"
The program is ready to run.  See icon on Start Menu.

On first-time launch, you are prompted for default backup destinations -- this is where you choose where you want the backups to write to.  See details earlier in this article, with additional program documentation here: Detailed Switch documentation



Version history:

2.02 - 2024.1005  Still had problems with high-resolution laptop screens; believe now fixed
2.01 - 2024.0923  Fixed display problem on laptops with high-resolution screens (140%)
2.00 - 2024.0810  Numerous design changes. 
          Improved Configuration selection and prompts
          Simplified Backup-type and option settings
          Added subfolder-cosmetic tagging
          Added drag-n-drop support
          Added eyecandy to help guide decisions
          Improved backend INI file settings

Note: Version 1.x INI config files not compatible with this version; sorry.  You will have to rebuild the config files.

1.05 - 2023.0430 Moved the Config/INI lookup button to top of panel
          Added cosmetic "Destination:" during the backup
          Added a "Delete" button (in addition to the Context menu) for INI Delete
1.04 - Fixed bug where sometimes root drive not inventorying: "Unexpected error in DOS directory"
1.03 - Minor changes to the report Log file, making it easier to find skipped directories
1.02 - Not released
1.01   2021.0610 Initial Release

Thank you to my Beta-testers: DLW, of Boise.

DirectoryPulse - Other documentation

 
DirectoryPulse! is a backup program for Windows computers and was written and tested with Windows 11.  The program should work with older versions of Windows.  

See this article for a general description and download instructions: DirectoryPulse Introduction

keyliner's DirectoryPulse program is free to download.  No registration.  No spyware.  No installation. 

DirectoryPulse works like this. 

• From the main landing page, add one or more top-level folders to backup
• Click "Refresh Directory Listing" to get a report
  
• Click the Gear Icon
• Use the Presets to assemble a destination path, and to pick the type of backup.
  
  
• Click Apply
  Optionally, click "Save" and save the preferences for later re-use
  
  
• On the main panel, click "Backup"

Saved Preferences can be re-retrieved, making a repeatable backup.  This can be scheduled and automated.  

First-Time/One-Time Recommended Setup:

On first-time launch, a "SystemDefaults" panel is displayed. 

The program was written with C#, .DotNet 8.0. When first launched, you may be prompted to install Microsoft's .DotNet 8.0 libraries.

A.  Change the default PRESET destinations to match your local drives.  For example, if you have a D:\ Drive, you can use it for quick, local backups.  The destinations are optional but recommended

There are three major types of destinations:

1.  DefaultLocal (typically Drive D:; can be an external USB drive)

2.  DefaultSAN (local Network Drive, Synology, etc.  Can be a USB drive. Disable with the word "none" -- or just ignore.)

3.  DefaultCloud (typically OneDrive or GDrive)

Type, browse, or drag-n-drop a root path for each destination -- pointing to a pre-built top-level folder, such as "D:\Backups".  Details below.

 

- Change DefaultLocal 

Set to a local backup drive.
Typically a "D:\" or USB drive. 
A root-drive plus a subfolder is required: for example:  D:\backup 

If a second drive is not available, build a dedicated sub-folder on the C: drive  (C:\backups\). 
(This is not ideal because it does not protect you from drive failures.) 

A DefaultLocal is required, even if this is a dummy location.

- Change DefaultSAN

Set to a Network-aware SAN drive, an external USB, or other resource for longer-termed backups.  Even though this choice says "SAN", any path can be used.  When pointing to this device, typically use a UNC\share-name or a mapped drive.  External USB drives can also be used. 

Examples:

DefaultSANBackupDestination  = \\SynologyNAS\Data\Backups   or
DefaultSANBackupDestination  = E:\Backups   or
DefaultSANBackupDestination  = \\PC-2\share\Backups  or
DefaultSANBackupDestination  = NONE to disable

The destination requires a root drive and subfolder (e.g. \Data, or Data\Backups).  Backups cannot be written to the root of a drive. 

Note:  if you are not authenticated to this drive, or the drive is offline, setting this field is slow as the program attempts to navigate and test the path.  Be patient.  The offline drive will time-out.  You can continue to build a non-existent paths, on the chance they are available later.

- Change Default Cloud  

Typically point to a OneDrive or GDrive local folder

Examples:

DefaultCloudBackupDestination =  C:\Users\<username>\OneDrive\Backups
"None" to disable

Gdrive's location is:  
DefaultCloudBackupDestination = C:\Users\<username>\AppData\Local\Google\DriveFS\backups

Replace <username> with your Windows 10 username or browse the Users folder (or leave text as a literal "<username>" and let the program determine the folder).  An active OneDrive (or GDrive) account is required.  A subfolder, such as "backups" is required.

Use the Cloud destination for offsite backups. When the backup job is built, the backup can (perhaps "should be") set with the [x] Zip option.  This improves file-transfer times and reduces disk quotas.  (Typically, these destinations do not use dated-folders or generations - just because of disk quotas; an option controllable during the backup.)

Continue with these other System Preferences:

d.  On the same System Default panel, the "Default Subfolder" field shows a comma-delimited list of recommended subfolder names.  These are cosmetic names that help build the assembled destination path.  Review the list, adding or subtracting entries. 

This is a comma-separated list, defaulting to:  Projects, Daily, Weekly, Monthly, Adhoc, Test.  Selecting "Projects, Daily, Weekly...." does nothing more than appends a common, predicable name to the already-typed destination path.  The assembled path can be overrode and changed by re-typing the destination's path. 

Click SAVE and close the SystemDefault panel.


During the Actual Backup
-------------------------------------------------------------------

Naming the Configuration Settings:

During the actual backup (the "Gear" icon -- see previous article for how this looks), the job can be named and saved. This way the same backup, with the same settings, can be launched at any time. 

There is an art to naming the configuration files -- name the files in a way that makes sense to you when looking at them a month later. 

Here are the names I am using:

Local-AllData-Dated.ini
Local-Util-Dated.ini
OneDrive-Projects.ini
OneDrive-Projects-Compressed.ini
OneDrive-ProgrammingBook-Compressed.ini
SAN-AllData-Dated.ini
Test.ini

When named and saved, the configuration files are saved as clear-text in
C:\Users\<you>\Prefs\DirectoryPulse\iniFiles

These are simple ascii text files and can be edited, deleted, and renamed at will from within File Explorer.  There is nothing magic here.  (The Config-file select icon also has a context-menu for manipulating the files within the program.)

Gear Details:
-------------------------------------------

Backup Types:

From the configuration panels (the gear icon), choose the backup type, where I typically use "Dated Folder":

 

Dated Folders:


Dated-folder makes a full backup of every file and subdirectory in the selected paths.  The entire backup's path and structure is re-created within the dated-folder. 

D:\Backups\myProject\2024-0801\(C:\)Data\subfolder\deeper-folder\way-down-deep   etc.

This type of backup allows you to go back in time and recover older versions of the file -- or entire subdirectories, but the backup is relatively slow -- having to make a copy of each file, each time the backup runs. 

The option, [x]Limit Dated Backups keeps 10 dated generations of backups -- ten versions of the entire disk structure.  On the 11th generation, the oldest folder is auto-deleted, making for a self-cleaning backup.  This is the type of backup I use the most often.  Be aware this type of backup can occupy considerable space on the backup drive.

Click for larger view

To preserve a backup (to keep it from self-cleaning), use File Explorer to rename the dated-folder to a different, non-date-like name.  For example:  Hold-2024-0801


Generational Backups:

This is the fastest backup and is similar to traditional "Differential Backups".  Only one folder structure is built.  The first-time backup is slow -- having to backup every file in the structure.  Subsequent backups only backup changed files.

When a changed file is detected, the older version of the file is renamed and the new version arrives -- first in the stack:

For example: 

myFile.xlsx      -- the most current version
myFile-#01.xlsx  -- the previous version (from the last backup)

myFile-#02.xlsx  -- the next oldest version

On the 11th version (the 11th edit), the oldest is discarded.

With this type of backup, a backup-subdirectory might contain 10 versions of the same file; each dated.

Because this backup only backs-up 'changed' files, it is a faster backup.  But because the generational-files "intermingle" in the same folder, recovering an entire folder is messier.  This is really meant for small-volume recovery.  Use this for "transactional backups" -- small data-folders with volatile files -- word processing documents, spreadsheets, and the like.

This type of backup can take significantly less space than the Dated-Folder backups.

Added benefits:  If a file is deleted from the original (Source) drive, older versions of the file remain in the backup-set and can be manually recovered.  In other words, older generations of the file remain "forever" in the backup. 

If the folder has 'lots-of' delete activity, the deleted files continue to occupy space in the backup.  The deleted files can be recovered by manually deleting the entire backup structure and starting over -- at the risk of losing other generational backups.  If you do this, make an immediate new backup as soon as possible.  Perhaps, some day I'll write an "orphan cleanup routine."

Zipped Backups:

Zipped backups are full-backups (no #generations), but the entire folder is backed-up as a ZIP file.  The Zip backup is typically used for OneDrive/GDrive/AWS backups and the zip-files take obviously less space and less time to transmit. 

For example:

C:\Data\subfolder1\50-files
becomes one zip file:  D:\backups\deepstorage\Data\subfolder1\subfolder1.zip

C:\Data\subfolder2\400-files

becomes  D:\backups\deepstorage\Data\subfolder2\subfolder2.zip

 

This is meant for archival backups.  In a disaster, recovery is by folder, one folder at-a-time.  This is not ideal if an entire disk is lost, but is useful for recovering relatively small number of individual files or folders.

-------------------------------------------------------------
Other switches:

[x] "Auto-Run"

Auto-Run is intended for command-line or Windows Task Scheduling, and is required for tasks called by Windows Scheduler.  This switch allows the backup to run unattended.  This switch means "run unattended" -- with no prompting, and the program auto-closes when done. 

Scheduling Backups:

Use the Windows built-in "Scheduler" to schedule periodic backups: See Microsoft documentation for details.  In summary:  Click Start, Run, "Task Scheduler".  Pass command-line parameters when launching, see next. 

Command Line Parameters:
DirectoryPulse can be launched from a command line with optional parameters to automate backups.  Or a desktop (shortcut) icon can be built with these same parameters.

By passing a previously-built preference file (ini file) from either a Shortcut, Windows Start Menu Tile, or from Windows Task Scheduler, tasks can be fully automated.  For example:

C:\Program Files\Util\DirectoryPulse.exe ini=MyFavoritebackup.ini

where: 

MyFavoriteBackup.ini is the name of a previously-saved DirectoryPulse Preference file
[x]Auto-Run should be (must be) flagged in the INI file

When launched with a Preference/INI file, the program loads, retrieves the preferences, scans the directories, and launches the backup.  When the backup completes, the program auto-closes.  Log files record the transaction.  Automation requires [x]Auto-Run as one of the switches inside the preference file. 


[x] Delete Path before Backup

This switch deletes the entire destination folder prior to running the new backup.  This is a brute-force switch and can only be used with "SIMPLE" backups.   Other backups, such as "Dated" backups, self-clean and do not need this setting.  With Generational backups, this switch is illogical and cannot be selected.

[x] Discard Cache Directories

Recommended.  Discards obvious cache directories in Firefox, WordPerfect, and other such programs.  The list of discardable directories and keywords is adjustable -- see the SystemConfig.ini file.

[  ] Allow System Files

Allows backups in directories such as C:\Program Files, C:\Program Files\Common Files.  This is not particularly recommended.  Using this switch will backup the files, if they are in the "SourcePath" list, but it does not backup Registry or System DLL's that might be required.  This is a typical restriction for installed programs.

[  ] Use last  DOS Inventory

Mostly used for diagnostics and this switch should not be used.  Under the hood, when clicking "Refresh Directory List", the program writes a DOS DIR - an ascii DOS Directory /s listing which shows the source paths.  A report is written as a temporary file (see users\prefs).  This checkbox says to keep the previously-built report and do not re-inventory.

Other files:

When first launched, DirectoryPulse.exe creates several files in
C:\Users\<username>\prefs. 

In here, find small control (.ini) files, optional reports, and log files.  In this same area, note the file "dirCreate.txt" -- this is created when "Refresh DOS Directory Listing" is clicked.  Of interest, this is nothing more than a DOS DIR listing, which uses this command:

dir (a directory name/*.*) /-C /N /oG /S >C:\Users\...\dirCreate.txt

This is the input file used by the backup program.  Basically, if DOS can see the file, DirectoryPulse can work with it.

SystemConfig.INI
Contains global settings, including destination paths, and discard directories.
This is a simple ascii text file and can be edited with care.
If you screw-up this file, see the nearby "controlFileBackups" folder or simply delete the file to start over.

Log Files:

ASCII-text log files are found in
C:\Users\(your name)\Prefs\DirectoryPulse\Logs 

The (15) most-recent logs are kept.  This count is not adjustable.

Easter Egg:  Hover the mouse here.

Restores:
 
DirectoryPulse uses simple file copies and the backups live in a folder-by-folder reconstruction on your source files.  To restore a file or folder, use File Explorer to copy that file/folder back to the source drive.  This program does not help in the restores.  Perhaps a later version.

To recover a generational file (e.g. testfile-#01.xlsx", copy the numbered version and then manually rename, removing the -#00 appendage.  Because some files may have more than one version, updated on different dates, restores are not automated.  

Other Comments:

DirectoryPulse is not an image backup and cannot be used to recover a crashed hard disk.  Instead, it is meant to keep operational copies of data, which can help recover from spreadsheet-blunders, ransomware, viruses, etc.

DirectoryPulse is not meant to backup Windows System folders, or ProgramFiles and it actively avoids these areas.  In any case, restores from program folders would be flawed because of Registry and other concerns. 

However, from the Settings panel, "[x] Allow System Files" will attempt to backup C:\Program Files, and similar folders -- if they are in the backup-list (Source Process Paths) -- but in no case will it fiddle in the C:\Windows folders.  This is by design.

OneDrive files:

DirectoryPulse sees and inventories OneDrive files -- especially if "MyDocuments" is in the backup path -- but this program avoids off-site pointers.  In other words, if OneDrive has a pointer to an off-line file, DirectoryPulse skips that file, even if it is in the backup path, and even if the file were marked as "changed."  (See Users\YourName\OneDrive).

The reason:  A backup of an off-site OneDrive file would cause it download and expand locally on your PC.  This could be gigabytes of network and disk activity.  DirectoryPulse assumes Onedrive is adequate and it will ignore the file.  

But, if the OneDrive file has already downloaded and expanded locally, and it is in the backup path, DirectoryPulse will back it up -- assuming the file has had a recent change.  This does not incur any extra overhead from the network or disk gods.

Backup Thoughts:

With any backup strategy, it is wise to make Offline or Near-Offline backups -- backing up to an external USB drive -- then disconnect the drive.  In the event of a ransomware attack (where every file, every drive, every SAN, is encrypted, that drive would be safe.  

Consider offsite backups.  Consider .ZIP backups.


Slow Backups to USB devices:

DirectoryPulse does nothing special when writing to the backup drive -- the technique is similar to an xcopy or robocopy. 

I have found some USB disks -- particularly "thumb drives" -- are painfully slow -- especially if the backup set is large.  The slowness is regardless if DirectoryPulse is being used or not.  This is a gnarly problem and seems best resolved by not using that type of device.

See this article for hints on how to improve this.
https://keyliner.blogspot.com/2010/07/acronis-2010-usb-drive-backup-speed.html

Preference File Location:
C:\Users\<login>\prefs\DirectoryPulse

Log File Location:
C:\Users\<login>\prefs\DirectoryPulse\Logs

--------------------
This program was a blast to write, and I use it daily.  I hope you enjoy using it too.  Your comments are welcome. 

Related Article:
DirectoryPulse Introduction
USB Drive Backup Speed Slow

NetGear Nighthawk AXE8700 Installation

How to:  Setup and install a NetGear Nighthawk AXE8700 Router (RAXE300)

Procedure: How to install a NetGear AXE8700 Wireless Router into an existing home network, replacing an older router.  This can be mostly done "offline" without affecting the current network, with a last-minute cutover, plus some delays for the Cable connection.

The setup can happen manually, without installing NetGear's cell-phone app, and without using wizards ("Genie").  If all goes well, existing wireless devices won't even notice the rug was pulled out from under them. 

This article assumes you are using a cable modem with a separate wireless router, illustrated as the NetGear AXE8700.  All wireless routers, all brands, behave similarly.

Prerequisites:

A.  A laptop or desktop with a wired RJ45 network cable/port  (these steps can also be done wirelessly,  not described here).

B.  A short network patch cable

C.  Able to login to your existing Wireless network router's admin panel.  The password may be printed on a label on the router, or if changed, refer to your own documentation.   If you do not have this password, and have no hope of finding it, continue with the article, with the understanding you will have to forget-and-reconnect all wireless devices.  This is not horrible.

Discover Existing Network

1.  Click once on the Windows 11 Start Menu and begin immediately typing "View Network Connections" (control panel)

2.  On either your network card or your wireless card, right-mouse-click and select "Status", then "Details"

3.  Write down the IPV4 address

This is the router's IP Address and it is almost always (always!) dot-one.  For example, mine is 192.168.200.1   (Yours might be 192.168.100.1, etc.)

4.  From any browser URL, type the IPV4 address, literally typing the dot-one address (for example: 192.168.200.1).

A login screen appears. 
Type userID "admin" followed by your router's configuration/login password.   

(If you cannot login to your existing wireless router, skip this step.  Consider opening your cell-phone's Settings screen, and look for "Network" -- note the broadcast Network names (SSID's) for your home network -- for example, mine are always called "Wolfhouse-".

Once logged in, select the "Advanced" section (varies by router, by manufacturer), and browse-around looking for the Wireless sections.

Note the Wireless SSID Names (case-sensitive)
Note the passwords (some router brands do not show passwords)

Typically default names are the unimaginative NetGear-5G,  NetGear-24G, NetGear-24-G-Guest, etc.  At my house, they are named "Wolfhouse-5G", "Wolfhouse-24G-Guest", etc.

Initial Hardware Setup

A.  Optionally connect the router to the Cable modem.

The new router can be initially-configured as a stand-alone box (off the Internet / not connected) and then dropped into place at the last minute.  Better if the router is near the cable-modem:  If so, connect from the cable-modem's yellow-out-port to the new router's second yellow-port, marked "Multi-Gig 2.5G/1G" -- use this even if your cable modem does not support the higher speed.  (To set-up the router without impacting the current wireless, leave this cable disconnected.)

B.  Connect your PC to the wireless router, LAN Port 1 (black port)

Assuming you have a wired Ethernet Cat-5 network port on your desktop/laptop, connect the network cable to LAN Port 1.  

On your desktop or laptop, temporarily disable any wireless cards (see the network control panel):

(If you do not have a hard-wired network port, I believe (but did not test) these configuration steps over the default wireless.)

C.  Power on the new router. 

Wait about 90 seconds or so (this router is slow to boot), then power-on your computer. 
(As an aside, your PC will acquire a new DHCP IP Address, similar to 192.168.1.3)

D.  Login to the router's administration screens

From your Windows computer, open Microsoft Edge or Chrome, type this dot-one URL:

192.168.1.1 

(Literally typing the router's default IP Address.  Later this address changes to 200.1)
Other brands may present as [192.168.1.1], or [192.168.100.1].  Use Edge or Chrome.  Firefox and other browsers seem to misbehave in some of the configuration screens.

Initial "Genie" (Wizard) login panel

1.  You will arrive at a Netgear setup page where it offers to install using a phone-app.  

Ignore the offer and scroll down the page. 
At the bottom, see the link "If you don't have a compatible mobile phone or tablet, click here".  Click the link and accept the lawyer-induced license agreement.  Sorry, no screen shot.

2.  At the "Wan Preference"

-- select "Multigig Port 2.5G/1G" -- selecting this port, regardless of the speed of your cable-modem.

3.  You may be prompted, "You are not yet connected to the Internet.  Do you want the NetGear Genie to help?"  Select "No, I'll do this myself"

4.  Invent a new admin password

Admin account:      admin  (case-sensitive)
Admin Password:   __________________

I recommend 12 or more characters - using a password phrase.  Write it down.
Complete the 2 security questions.
Important:  Write down both the security question and the security answers*

*Reason:  As of 2024.07, NetGear has a bug.  If you forget the admin password and it prompts for the security questions -- it does not show which security question you chose!  It says, "what is the answer to question 1" -- not helpful; a bug!

5.  If the router is wired to the Cable Modem, [X] Check "Smart Connect".  Otherwise, leave unchecked.

6.  Wireless SSID names and passwords

When prompted, set the 2.4G, 5G, and 6G SSID name's and passwords, following the on-screen prompts.

Ideally use the same SSID names from your old router (e.g. Wolfhouse24G, etc.).  Within each type of wireless network, I typically set the 5G and 6G passwords the same, and I set the slower 24G password differently -- this is what I tell the kids -- tee hee.

The 6G connection may be new to your network.

Note: Once complete, the next page shows a report and that report may show the wrong SSID names (showing the default "NetGear-24G" names and passwords); safe to ignore.  Be aware the "Guest" networks are not yet defined.

7.  Allow the Firmware check -- which may or may not find an update and depends if the cable-modem is connected.  Safe to temporarily ignore.

"Setup Genie is complete" (even though you didn't use the Genie steps). 
After the Firmware check, you arrive at the normal setup panels.

Advanced Settings - Recommended

Click the "Advanced" Tab

1.  Change the Router's main IP Address to match the old router's setup.  

For example, my old router was at 192.168.200.1.  Yours may have been 192.168.100.1.  (Out of personal preference, I set my main network at a different default address than the out-of-box addresses.)

In the left-Nav's "Setup" section, "LAN Setup"
Edit the IP Address, changing from 192.168.1.1  to 192.168.200.1

Save the changes.
Your desktop/laptop will lose its connection to the network and the Setup panels will (close).

- Restart your desktop/laptop to acquire a new IP Address
- At a browser, type this new IP Address (192.168.200.1) to re-open the Setup panels:

192.168.200.1

Again, login as "admin"

If the admin login fails (because you have bad short-term memory, and didn't write the password), power-off the router.  With a pen, press and hold the micro-Reset button on the back of the panel.  Power-On, holding that button for 8 seconds.  Start over with the setup, following the on-screen instructions.  Re-do all the steps above.

2.  Change the default DNS servers

In the left-Nav's "Setup" Section, "Internet Setup", make this recommended change:  Change from your ISP's DNS server to a safer default, choosing a new Primary DNS:  Google's:  8.8.8.8.  Scroll-down to find this setting.

Optionally set the secondary DNS to your ISP's value, here illustrated as CableOne's 96.dot address.  For my own networks, I set both the Primary and Secondary address the same, bypassing the ISP's DNS.

You must click "Apply" to save the changes.

If your home network has a Raspberry Pi "Pi-hole" DNS sinkhole (a fun project, see this link: Stop Advertisements using a Raspberry-Pi Pi-Hole), temporarily leave the ISP's default DNS address.  Later, after the PI is online, return to this menu and type the PI's address.  Reason: While setting up, the Pi-hole may not yet be on your new network.  Don't point to it until the router is running in production.  Another reason to delay setting the Pi's IP address is probably within the router's auto-assigned DHCP range and collisions will occur.

3.  Adjust the DHCP range

The router can auto-assign floating IP addresses to other devices on your network, using a feature called "DHCP".  Reserve IP addresses below 10, and above 150 (or as illustrated, 100) for hard-coded devices such as an in-house web-server, a Pi-Hole, printers, and TV's.

Return to the "LAN Setup" panel, scroll-down
Change the Starting IP Address to .10 
Set the Ending Address to .100 (better to use 150)
"Apply"

 

If you get an error, "The IP Address conflicts with the WAN IP subnet. Please enter a different IP address", it is basically saying "I can't set these ranges because another device is using one of those addresses already."  Solution:  Temporarily get those devices off the network by either unplugging their Cat5 cables or powering them off.  You may need to reboot the router to flush the cache.

4.  Confirm Wireless

On the Advanced tab, "Wireless" section, 

Review each of the available wireless networks and their passwords.  

With this router, there are 3 different production networks and these were set in an earlier step.  It is wise to review them again:

NetGear-24G (Wolfhouse24G)
NetGear-5G   (Wolfhouse5G)
NetGear-6G   (Wolfhouse6G)

Again, if the default SSID is "NetGear-this-or-that", change the name to something like "Wolfhouse24G", "BatHouse", "FBISurveillance", etc.

On this particular router, these screens are confusing because it is hard to tell where one wireless network setting ends and the next begins.   Each wireless section has various other options (such as WPA encription, etc.).  It is safe to accept the defaults -- you are mostly after the passwords.

Click "Apply" to save the changes.


5.  Confirm "Guest Network"

Similarly, review each "Guest" network.  In the left-Nav, select "Guest Network".  Make the passwords different than production networks.  I recommend only exposing the slower 24G network.

WolfhouseGuest24G  (Enable and broadcast SSID)
WolfhouseGuest5G   (Disable and do not broadcast SSID)
WolfhouseGuest6G    (Disable and do not broadcast SSID)

6.  Final Wiring

* You do not need to contact your Cable Provider to tell them about the new router *

Power down your original/old wireless router
Install the new router, as illustrated at the top of this article
Return your workstation to the normal network (the Switch), LAN-Port 2, Wireless, etc.  (If you previously disabled the workstation's Wireless card, re-enable it.)

Power-up in this order:
a)  Confirm the Cable Modem is powered up (you do not need to turn it off during this upgrade). 
     If rebooting, wait 1 minute before next steps.

b)  Power the NetGear Wireless Router 
     Wait 1 to 2 minutes for the router's status lights to turn white

c)  From your normal workstation, use a browser to test Internet connectivity, browsing Google or some other external site.  If it can't find the network, reboot the workstation to get a new DHCP address.

d)  Test your network printers.  Reboot the printers if necessary.

Possible Problems:  

The wireless router looks good.  The new router's dashboard (Network status panel) shows a good external IP address -- but no devices can see the Internet.  

Solution:  I had this problem and did not find a satisfactory answer.  In the end, I waited an hour or two, then rebooted both the Cable Modem and the wireless.  I suspect a MAC address needed to flush from the ISP's cache.  This seemed to fix my problem.  Remember, you do not need to call the Cable-company -- they do not care about your new router.  

The other possibility is to re-connect to the Router (typing 192.168.200.1).  At the Router's left-Nav, click the Setup Wizard.  This will keep most of your previous settings, but will make you re-build the wireless SSID's and passwords.  Hopefully, the instructions on this page are good enough and you can avoid this step:

Final Cleanup

Consider and test these other devices that might be on the wireless network. 

Laptops
Cell Phones
Tablets
TV
SAN drives
Thermostats (NEST, etc.)
Security Systems
Water Detectors
Light Switches
Garage Door openers
Sprinkler Systems
Can Openers (joke)

If the device does not find the network, a reboot should fix the problem (provided your SSID names and passwords are the same as before.  If not, have each failing device "Forget" their network and re-login.)

Rare Items:

Consider these esoteric items that I have on my network:

Rare:  If you have a Raspberry-Pi Pi-hole DNS Sink Hole, re-login to the NetGear Wireless router setup screen (192.168.200.1).  Change the DNS from your ISP's address (or from 8.8.8.8) to your Pi-Hole's address, removing the previously-set addresses.  See article, below.

Primary DNS:  192.168.200.151  (my Raspberry-Pi's hard-coded IP Address)

Secondary:  192.168.200.151  (repeated, or 8.8.8.8) 

Rare:  If you have a local https webserver running inside your network, it needs holes punched through the firewall for ports 80 and 443.  See article, below.

Rare:  If you have a local webserver, check your external-facing IP Address.  It likely changed.  At your Domain Provider (Godaddy, etc., likely different than your ISP), review your "A-Record's" IP-address.


Final, Final Cleanup:

Use a Brother Label Printer and print a snazzy label, showing the Router's IP Address (dot-one) and its new administrative password.  It might be helpful to also print the 24G Guest password.  Stick these on the new router.  (Then, remove the factory password label).  Better yet, document these settings in a notepad document.

After a few days, once you know all is well, wipe-out the old router's memory:  With the old router disconnected from the network, power-up the device.  Find the micro-reset button, found on the back of the device.  With a pencil-tip, press and hold for about 10 seconds, then power-off.

Other Related keyliner links:

Install your own Webserver
Stop Advertisements using a Raspberry-Pi Pi-Hole