2023-08-30

VNCViewer Timed out waiting for a response

Problem - VNC Viewer Timed Out waiting for a response from the host computer
Problem - VNCViewer client not responding

Windows 11 client, and a Raspberry Pi.

Article under construction.  Here are my notes.
An imperfect solution is listed below.
Consideration #3 has helped a lot of people, but was not a solution for my problem.

Symptoms
Desktop computer attempting to reach a VNC Server (in my case a Raspberry PI, acting as a server) gets ~ error

VNC Viewer 'worked before' or has worked "occasionally", or works one-time.

The host computer (the server) is operational and VNC is installed properly
The server is ping-able
VNC services are running

Waking the host-server (from sleep) did not solve the problem
Some sites recommended plugging in a monitor, keyboard, and mouse; nope; did not help

Diagnostics:

Confirm the service is running on the server:
From a remote SSH desktop connection (Putty.exe), run this console command:

sudo systemctl status vncserver-x11-serviced.service
(then ctrl-C to cancel/close the command)



Look for:
"Active (running) since <date>"  (green)
"Started VNC Server in Service Mode daemon" and
"ServerManager: Server started."

* You may see:  "ConsoleDisplay: Cannot find a running X server on vtl"
Other web articles suggest VNC will not operate without a monitor plugged-in and active. Nonsense. Many Raspberry pi's are "headless" -- without a monitor or keyboard.  Adding a monitor, keyboard, and mouse is not the solution to this problem.  This idea can be ignored.

Assuming VNC was installed properly; services started, IP Address ping-able, and all that nonsense, it can't be a problem with the server or with the VNC Server Services not running.  It can't be a problem with ports, firewall rules, or other configuration -- after all, it worked before...

Poor Solution:

This is a less-than perfect solution.

1.  From the Windows VNC Viewer Client, delete the Connection icon to your server
     (e.g. "keyliner.local") -- this is the connection that worked before, but does not now.

2.  Rebuild the connection from scratch.


This is asinine and indicates a possible problem in C:\Users\<you>\appdata\local\RealVNC folder

But after hours and hours of research, a proper solution was not found.


Considerations:

1.  Be sure to upgrade VNC and other services on the Server's desktop:

Even a new install, with a new distro, VNC needs to be updated.  From a terminal/console command window (putty.exe), or from the Raspberry pi's Terminal (console),

type sudo apt update
type sudo apt upgrade

(I noted the VNC service was updated during this patch cycle)

2.  Some articles recommend forcing a manual screen resolution

From a terminal/console command window (putty.exe), or from the Raspberry pi's Terminal (console),
type  sudo raspi-config
Select "Display Options"
Set "D5 VNC Resolution" to a fixed setting, such as 1280x1024  -- anything smaller than your Windows client's native desktop size.

This does not fix this immediate problem, contrary to other web articles.  However, setting a predictable resolution seems a good idea.


3.  Many have reported a bug in the Raspberry PI operating system, where a "headless" pi (one without a keyboard/mouse) does not generate a 'cached random-number pool' of some type.  (However, my pi has a keyboard and mouse installed (at least temporarily) and this did not help.  I discounted this idea.)

To solve this random-number problem, some sites suggest installing a program called "haveged".  Reportedly, this fixed VNC for many users, but it did not resolve the issue for me.  Regardless, here are the full details and credit in this article:
https://forums.raspberrypi.com//viewtopic.php?f=28&t=230779&p=1413648#p1413648

Not-recommended steps:
sudo apt-get install haveged
sudo update-rc.d haveged defaults


(to uninstall):
sudo apt remove haveged,
followed by sudo apt-get --purge remove haveged  )


Bad-to-Worse Solution:

Start a new VNC session: 

This is not necessarily a good idea, but will work (one) time per boot -- starting a second virtual desktop on the server -- a second copy, separate from the already-running service.  This seems a poor solution.  In any case, from the terminal/console window, type this command, launching a second VM (desktop)

vncserver

Test by launching the client's copy of VNC and make the VNC connection.  This should should work, but once you disconnect the client's VNC, you will not be able to make a second connection without rebooting the pi server.  Spinning-up a second instance of a VNC server masks the underlying problem.


Other notes

Installation


Starting and confirming the VNC Service on the Raspberry pi server:
Assuming VNC was not started or installed on your raspberry pi server, and assuming SSH was installed...

a.  Using Putty.exe on a Windows box (remote console)
     (Download from:  https://www.putty.org)

     Launch and connect to your server, from the local network (e.g.  my server: "keyliner.local")
     Login as  admini
     and the SSH password: 
       
b.  Type this command:  sudo raspi-config

c.  Select menu choice "Interfacing Options"
     Select menu choice I3:  "VNC"  Enable/disable; enable the service.

d.  The Windows VNC Viewer (client) can be launched in either a local, non-logged-in mode (recommended) or with a "cloud" (internet) account.  Both exhibit the same problem and I see little reason to build the RealVNC "cloud" account.

I have recommended bypassing the cloud login:

On your (Windows) workstation, download and install a VNC Viewer client.
     Download:  https://www.realvnc.com/en/connect/download/viewer/
     Select Standalone EXE x64 Installer

     On startup, note the link:  "Use RealVNC Viewer without signing in"
          (assuming a local network, not over the Internet)

     Connect to your server's name:  e.g.  "keyliner.local"


In Case Needed:  To Start the VNC Service:

As an alternative to raspi-config, from the remote console:

a.  sudo systemctl start vncserver-x11-serviced.service
     (where x11 is x-one-one.  There is no console response to this command)

To Enable VNC Service at next boot, and for every boot:
a.  sudo systemctl enable vncserver-x11-serviced.service


Resources:
Article on everything about VNC on Raspberry pi:
https://help.realvnc.com/hc/en-us/articles/360002249917-VNC-Connect-and-Raspberry-Pi#creating-and-remoting-a-virtual-desktop-0-8

-end


2023-08-21

Device ID Asset Tag Management

DeviceID is a computer-tagging program that shows the machine's current IP Address, device name, user, and network status.  In the background, an inventory is written to a local report and optionally to a server, which the helpdesk will appreciate.  The program sits unobtrusively in the upper-right corner and periodically checks network conditions. 


Network status checks both internal and external IP addresses, confirming the machine can see the Internet.  Settings are adjustable and every object on this small panel is double-clickable, giving different reports, different features. 

Behind the scenes, DeviceID tracks an inventory of the hardware, free-disk space, OS settings, recent logins, and other interesting goodies.  If the report is allowed to live on a central server/share, the helpdesk can view activity, even if the user's machine is offline.  Inventory is tracked through simple ASCII files; no database required.

Free to download and use for all personal, commercial, and governmental use. 
* No registration
* No nagware
* No spying
* DeviceID does not call back to the mothership

Features:

  • Network status is periodically tested against internal (company) and external (Google) sites.  Red-green, yellow flags show the current status.

  • An inventory report shows the OS version, free-disk, memory, installed printers, CPU, the last 10 logins, etc..

  • Reminds users to reboot every (x) days.  Configurable from a central location. 

  • From a central server, the program can optionally run scheduled tasks, installation, and other routines. 
    This is not meant to replace a full-featured software installation server, but could be handy for a smaller business.

  • From a central server, emergency messages can be sent.  Alerts, such as "Database maintenance today at noon."  

Although this is a long article, and it seems complicated, it really only takes a minute to set up.  The advanced features can be ignored.  Although designed for small to medium-sized businesses, the program is useful on standalone machines.  I run this program on my home network, mainly for IP and network status.

Downloading and Installing:

Here are the fast installation instructions where the program can be run directly from the Downloads directory.  It needs a minor installation step.  
 
To install, download the .exe and place in any temp folder on your hard disk. 

 
A.  Download Link:

From Keyliner's public GDrive, click this link and download to a local temp or download directory.  Do not download directly into ProgramFiles. 

Download Link: Version 1.30 (New, 2023.10)
File Link:  DeviceID.exe - contains standalone .exe.

deviceID.exe MD5 check:

a5-d6-6b-f7-37-97-30-68-a4-1f-34-17-f6-25-24-c3

SHA-256:
d93b32d976492870ab99559b2378c3e11fc21c3f6295b3e1409eac68d7bd1d50

When downloading, different browsers behave differently.
You will be prompted the file cannot be scanned.  Click "Download anyway"

Microsoft Edge:
Prompts "DeviceID.exe" was blocked because it could harm your device (it is an .exe from the Internet). 

"Click See More" and allow the download. 

With Microsoft Edge, the downloaded file appears in your Downloads directory with a random name, such as "Unconfirmed 780359.crdownloaded" (name varies).  Use File Explorer to rename the file to "DeviceID.exe".

B.  Mark the program as safe-to-run:

(This step may not be needed if downloaded by Edge and you clicked "More / Download Anyway")

Using File Explorer, right-mouse-click the downloaded (and re-named) .exe 

Select "Properties"
Check [x] Unblock.  This removes the "mark of the web." 

                 Click for larger view

* Only do this if you trust keyliner *and* only if downloaded from keyliner's public GDrive. 

If "Unblock" is not visible, it has already been unlocked (by Microsoft Edge).
Once [x] Unblocked is clicked, this security menu disappears.

C.      For a first-time run, launch with a setup switch
Click "Start" and immediately type the word "Run" (enter).  A dialog opens.

From the Run box, use File Explorer to click and drag the downloaded "deviceID.exe" program into the run box (illustrated below).  Append a "/setup" switch and allow the program to run: 
DeviceID.exe /setup


Since keyliner cannot afford a signing certificate, you will be prompted that the file is not safe (being downloaded from the internet).  Click "more information" and allow the program to run.

Choose "[x] Server simulation"
Click "ok"

This writes two INI control files to the same directory.  For the initial test, they can be ignored.  Details below.


D.      From the same directory, re-launch DeviceID.exe (without the /setup switch)
The program should be running.
See panel in the upper-right corner of the primary monitor.
 

Details:
/setup builds two .ini files, both stored in the local directory. 
The default settings will run as-is, which is good enough for initial testing:

C:\downloads\DeviceIDLocal.ini   (where the .exe was stored)
C:\downloads\DeviceIDServer.ini 


Using Notepad, you can edit the ini files to see what they are up-to. 

The local ini points to the server file and to an optional user-specific 'reminders' file (a calendar).  The Server ini describes shared network resources, messages, etc, and is intended to live on a shared, centralized server.  The server file contains global settings, a place to put messages, and other settings.



For a more professional installation:

To install the program 'more properly,' follow these slightly more advanced steps.  Use these steps for a more formal installation.

Setup

1.  Server Share Setup

If using a central server (typical for a business, but not for a home user), make a dedicated server folder, typically a server share (It helps to pre-build these directories, but DeviceID will attempt to create them, provided your account has rights).  If not using a server, select option [x] ServerSimulation.

For example:

P:\Software\Util\DeviceID\wsConfig     //A server share
   (use any path - system .ini files live here)

P:\Software\Util\DeviceID\wsInventory 
   (workstation inventory reports live here)

Set end-user rights: 
If using [x] ServerSimulation, ignore this step.
Set all users to "Read-only" rights to the top-level "util\DeviceID" folder, cascading down. 
Set all users to "Change" rights at the wsInventory. 


2.  Complete the initial download, described above.

Place the .exe in a temporary working folder (download folder is fine)

3.  From the download directory, temporarily, launch the program using the /setup switch

a.  Click Start, type the word "Run".  Launch the Windows "Run" app.
b.  In the small Run box, use File Explorer and drag "DeviceID.exe" into the box.  (Or browse)
c.  In the RUN dialog, after the .exe, type {space} /setup
d.  Click OK and allow it to run

4.  Build Default Settings and Folders

From the Setup panel, which opens after typing /setup:
Browse to a server folder (P:\software\util\DeviceID) where the Inventory and ServerConfig files can roost - this is typically a server share. 

From the setup screen:

Click for larger size

Ideally, this folder should be pre-built.  If not, DeviceID attempts to create the directories, provided your account has rights.  If you don't have rights, it will still record that path in the ini-control files, assuming you will fix the missing path later.

If testing on a stand-alone workstation, or if testing without using a server, check
[ ] SeverSimulation
(recommended if you just want to play with the program and not make a commitment -- see C:\Temp\ServerSimulation\DeviceID).

Click button "Build ini Files"


5.  Results


New Directories:

a.  C:\Users\(you)\prefs\DeviceID   for workstation and user-specific data
b.  C:\temp\ServerSimulation\DeviceID
c.  C:\Temp\ServerSimulation\DeviceID\WSConfig  (all users have READ only rights)
d.  C:\Temp\ServerSimulation\DeviceID\WSInventory (all users need Create/overwrite rights)

New INI files are built:

File and folder schematic:

Click for larger view

DeviceIDLocal.ini

Distribute the .exe and this INI file to any workstation. 
Typically copied to C:\program files\Util  or location of your choosing, placing both files in the same directory.  For testing, these can be in the TEMP folder.
The .INI contains a pointer to the server's config file.

(End-users are not expected to have rights to edit the INI file and this is why ProgramFiles is a good choice.)

DeviceIDServer.INI

(This second INI file is a server file.  For testing, it can also be in the TEMP\ServerSimulation  folder)

The first INI file (DeviceIDLocal) tells the workstation where to find the server's INI file. The server INI file contains all the settings you want the workstations to share.   Plus other server-administration settings, such as global messages, etc.

Both INI files are editable with notepad and you are invited to explore. 

Next, position the program and icon as-if it were in production.  (This is a continuation of the "more professional install steps)


5.   Create a Program folder to hold the program:

For the local user, end-user workstation:

a.  With File Explorer, open folder C:\Program Files,
b.  Create a subfolder  C:\Program Files\Util  (for example)

c.  Copy the program file:

Copy "DeviceID.exe" to C:\Program Files\util
Copy "DeviceIDLocal.ini" to C:\Program Files\util

Do the copy as a two-step, copying from the download folder into Program Files.  Windows security will not let you download directly into Program Files (technically, you will not be able to remove the "mark of the web" if downloaded directly into Program Files).

DeviceIDLocal.INI is purposely in C:\Program Files.  Reason: Most end-users (should not) have administrative rights and won't be able to edit the file.  This is by design.  This forces their connection to the server-share.

6.  Position the Server Control file

If using a server
Copy DeviceIDServer.ini  to P:\software\util\DeviceID\wsConfig

If not using a server ([x] ServerSimulation), leave the folders and files as-is in the C:\Temp folder.
(See DeviceIDLocal.ini for the location being used)


7.  Optional: Create a Start Menu Tile:

Using File Explorer, in C:\Program Files\util
Right-mouse-click the executable and "Pin to Start"

Auto-launch steps below.


8.  Auto Launching:

Allow DeviceID to auto-launch at boot.  Place a shortcut in the workstation's "Startup" folder.  Optionally, put the program in a login script (not recommended because the program will not run off-network; it is better to run at the workstation level):

a.  Using File Explorer, tunnel to
C:\Users\<you>\AppData\Roaming\Microsoft\Windows\StartMenu\Programs\Startup

b.  In the Startup folder, right mouse click a blank area on the detail side, "create shortcut".
Set the path to C:\program files\util\DeviceID.exe


Testing

With setup complete, and assuming the ServerSimulation (or server directory),

- Confirm DeviceIDLocal.ini is in the same directory as the .exe. 
- Use File Explorer to double-click the .exe, which launches the program.

Expected results:

a.  Program loads in upper-right corner, Network Status = (green)

b.  Folder C:\Users\<you>\Prefs\DeviceID has an inventory report.  On my computer this is called "Oscar.txt", where Oscar is my workstation's Name.

c.  Using FileExplorer, see folder C:\temp\ServerSimulation\DeviceID\WSInventory for the server version of this same inventory report.  Useful for helpdesk research.

d.  On the running program, double-click the workstation's name for small report.

e.  On the running program, double-click the word "DeviceID" for full workstation report

f.  Optionally: On the running program, double-click your user-id.  Allow it to build a reminder file, such as "Bob's birthday on 08/22". 

g.  Hover mouse over IP Address to see the external-facing IP address.


Off Network Workstations

If the workstation is off network (and it can't detect the Server.ini), the program disables all server-related activity.  The local intranet test site temporarily changes to the nearest router, detected automatically (e.g. 192.168.1.1).

The External Test Site temporarily defaults to www.google.com, in case your default pingable server was different (these preferences are on DeviceIDServer.ini).  Colored status indicators continue to work.

The local inventory is still written to C:\Users\<user>\prefs
The server inventory is skipped.


For Documentation:  Generated INI files:

Example Local.ini

(at each workstation)

;DeviceID Local INI Control File
;Directs DeviceID to the ServerControlFile; must be in current DeviceID.exe directory
;Position-dependent file; do not change structure or line counts

ServerControlFile             = C:\temp\ServerSimulation\DeviceID\WSconfig
                                                         \DeviceIDServer.INI
LocalReminderFile             = C:\users\<username>\prefs\DeviceID
                                                          \DeviceID_Reminder.ini
WorkstationOffsiteFlag        = N
LocalDisableRebootNagSW       = N
LocalDisableRunEventsSW       = N
LocalHorizontalOffsetPixels   = 0
LocalVerticalOffsetPixels     = 0


Example Server.ini

(At the server's WSConfig folder)

;DeviceID
;ServerControlFile DeviceIDServer.INI; Do not delete
;Note: Workstations have a local file that points to this file
;Example location:  \\Server\Share\Util\DeviceID\Config\DeviceIDServer.INI

[General]
ServerInventoryDirectory              = C:\Data\proj\util\DeviceID\serverSimulation\DeviceID\WSInventory
DefaultLocalReminderINI               = C:\Users\<username>\prefs\DeviceID\DeviceID_Reminder.ini
ActiveDirectoryLDAPPath               = cn=prod,dc=mycompany,dc=com
DefaultHomeDataDirectory              = H:\Data
PhysicalLocation                      = Main Campus

InternalTestSite                      =
ExternalTestSite                      = www.google.com
ExternalIPAddressTestSite             = https://api.ipify.org  ;Shows external-facing IP address

RebootNagWarningDaysYellow            = 5
RebootNagPromptDaysRed                = 7

[Launch Run Events]
;This event can be set to run on program launch.  This is a one-time event per launch.
RunAtLaunch                           = N
RunAtLaunchCommand                    =
RunAtLaunchParameter                  =
RunAtLaunchMarkerFileName             =

[Emergency Run Events]
;Run these events in an Emergency with an optional user prompt
;Caution: RunLine is and can be called multiple times; manage duplicate runs with Run Once Marker Filename
;When used, Usually RunAtLaunch=Y, RunAtHourlyCycle=Y, with marker filename, for fastest results
EmergencyRunAtLaunch                  = N
EmergencyRunAtHourlyCycle             = N
EmergencyPromptMessageHeader          = Important Message:
EmergencyPromptMessageLine1           = This is an example emergency message
EmergencyRunCommand                   = notepad.exe
EmergencyRunParameter                 =
EmergencyRunMarkerFileName            =


Set an example emergency message:

Optionally: The program can connect to a server to show simulated company-wide messages or to run company-wide programs.

With your copy of DeviceID still running,

Edit C:\Temp\ServerSimulation\DeviceID\WSConfig\DeviceIDServer.ini

- Set "EmergencyRunAtLaunch = N   (no need for this test)
- Set "EmergencyRunAtHourlyCycle = Y
- Set "EmergencyPromptMessageLine1 = "Database maintenance tonight"

Leave all other Emergency fields blank. Save the file.
Results:  After roughly an hour, and then bugged every hour after that, users will see the database maintenance message.

Edit the file again. 

Set "EmergencyRunMarkerFileName" to something like: 
2023.0828-DatabaseMaint.txt  (any made-up name). 
Save the file.

Results: After another hour, the user is bugged once, then never again

Marker files are stored per workstation.  See C:\Users\<you>\prefs\DeviceID. 
Delete the marker file and they will be nagged again. 
Recommend using unique marker file names with any Emergency Run. 
Only one Emergency allowed at-a-time.

The hour delay is to keep the server from being overwhelmed if everyone logs in at 8:00am and the delay is randomized.


Testing Comments

Because of the ServerSimulation, WSInventory will only have one workstation -- yours.  In real life, all workstation writes to this directory.  Fifty workstations = fifty files. 



I would appreciate hearing about your experiences with this program.

Version History:
1.10 Initial Release
1.11 Generates inventory report on demand, not just at launch.  Double-click label. Ping testing less aggressive if all "green". 
1.20 Added External-facing IP Address reporting.  Click once, then hover on displayed IP Address.
1.30 Added OneDrive backup for Reminder.INI file; Reminders now move across machines via OneDrive.  If OneDrive not available, feature disables.


Related Keyliner Programs
Midy5 Checksum finder
Directory Pulse Disk Management

The program is a free keyliner-developed application, with no registration, no sign-in, no ads, and does not call the mothership.  Write this program yourself!  This started out as a Student project in my book War and Peace Programming Volume 6 (Visual Studio C#) - Project 5.     


2023-08-14

Error - Thunderbird - Your message was sent but a copy was not placed in your sent folder

 Error - Thunderbird - Your message was sent but a copy was not placed in your sent folder

"Your message was sent but a copy was not placed in your sent folder (Sent) due to network or file access errors.  You can retry or save the message locally to a local folders/Sent."

Plus: "Status: Copy failed"

Problem:
When sending an outbound email
Likely, you had just recently completed other Outbound SMTP email changes or the message you are trying to send was previously written before SMTP changes were completed.

Solution:

At least for me, the solution was to close and re-open the Thunderbird client.
Then re-compose a new test message.

(and, I am assuming you are using IMAP, where message are synced to the client and to the server - recommended.)

This fixed my problem.
Other articles recommended rebuilding folders, going into the Thunderbird's settings and changing how Outbound emails are stored (storing them locally), turning off Oauth2, turning off TLS, and other nonsense.

Error - Thunderbird SMTP login to server smtp.office.365.com failed

Error - SMTP login to server smtp.office.365.com failed
Error - Unable to authenticate to Outgoing server (SMTP) smtp.office.365.com.  Check the password and verify the 'Authentication method' in 'Account Settings | Outgoing server (SMTP)

Summary: 
A change is required in a default O365 setting.
This example uses GoDaddy, which is hosting my O365 email.  Your email provider will have similar steps.

(O365 / Office.com will not host your email directly, they always go through a third-party)
Note:  O365 has been renamed to "Microsoft 365".  Microsoft has a branding problem.

Symptoms:
Thunderbird email client (version 115 and newer) displays ~ messages.
Receiving emails is not a problem. 
Error appears only when sending.

This can happen with other email clients, other email providers.
The email client must be capable of logging into O365 new OAuth2 authentication methods (with Mozilla Thunderbird, version 85 or newer). 

Solution:

Assuming you are using IMAP email.
Assuming you are using GoDaddy.com (suspect other email providers follow similar steps)

Confirm these SMTP email settings

Likely already correct:

A.  In Thunderbird, "Account Settings", "Server Settings"

     Confirm:  Server Name:  outlook.office365.com   Port 993
     Confirm:  Connection Security:  SSL/TLS
     Confirm:  Authentication method: OAuth2

B.  In "Account Settings", "Outgoing Server (SMTP)"
 
    "Edit"
    Confirm server name:  smtp.office365.com
    Port 587
    Connection Security: STARTTLS
    Authentication method:  OAuth2

    Note: By design, with OAuth2, there is not a typed password

Change this O365 Email setting:

1. Login to GoDaddy
2. At the Email Dashboard, in "Account Information", click "Advanced Settings"

3.  Enable (turn on) "SMTP Authentication"

4.  Important:  Wait (apx 24 hrs) before this change takes effect.   Yes, you must wait.

5.  Testing:  The next day, open Thunderbird: Send a test email.


Other sites, other articles, recommend turning off SSL/TLS.  No.  Don't do that.  Others recommend using a user-ID/password for the SMTP login (turning off OAuth2).  No, don't do that either.  While these workarounds will solve the problem, they miss the point -- fixing the symptoms, and causing other security problems.

You will not need an Windows app password, as suggested by other sites.

I would appreciate comments on how to do this with other, non-GoDaddy email providers.