File: android-tkinter/CODE/launch-mergeall-GUI.pyw

#!/usr/bin/python
"""
================================================================================
launch-mergeall-GUI.pyw:
  desktop GUI launcher (part of the mergeall system)   

-Copyright and author: © M.Lutz, 2014-2019 (http://learning-python.com) 
-License: provided freely, but with no warranties of any kind

#-------------------------------------------------------------------------------
# ANDROID VERSION, January-April 2019
# Per http://learning-python.com/mergeall-android-scripts/_README.html#toc8.
#
# Replace original file with this custom version on your phone (only).
# See "# ANDROID" for all changes applied here and in mergeall_configs.py.
# These changes are for Android only, do not impact the command-lines mode 
# of Mergeall, and might be merged into the original file in a later release.
#
# Recent changes (search here for their [date] labels to find changed code): 
#
# [Apr2119] Pydroid 3 3.0 broke webbrowser: use os.system(cmd) with a 
#           hardcoded Android activity-manager command line instead
#           (3.0's $DISPLAY breaks module, $BROWSER kills "file://").
#
# [Apr1919] Fix merge-spawn workaround to the Pydroid 3 empty sys.executable 
#           bug, to accommodate the different Python path in Pydroid 3's 3.0.
#           The fix reads a spawned 'which python' command to be path agnostic.
#           Also go back to using webbrowser for help and logfiles: it does 
#           work, but iff files use '"file://" and HTML docs use online URLs;
#           see _openbrowser.py for a demo and more details.
#
# [Apr1219] Reenable help and popup-logfile buttons, and fix them to open with
#           an os.system() spawn of an activity-manager command instead of Py's
#           webbrowser; open online version of help, for its latest changes.
#           Also use default color for "Help" - tkinter loses a custom bg.
#
# [Mar2819] Shorten label text again and further, for smaller phone displays.
#           Labels now fit >= 5.5" screens; if still too wide, set the new 
#           LABELFONT user config, or search for "# ANDROID - shorter" to edit.
#           Also for smaller displays, resplit run-dialog text (but not paths).
#
# [Mar2319] Made message text readonly to avoid keyboard popups on slow swipes.
#           This precludes copy/paste, but the on-screen keyboard is annoying.
#           Also manually split run-dialog text (but Tk still truncates paths).
#
# [Mar0819] Added folder-chooser prefills and starts as config-file options.
#           These can often avoid the tedious Android Tk folder-chooser dialog.
#-------------------------------------------------------------------------------

A portable Python 3.X/2.X tkinter/Tkinter desktop GUI, for easily launching
mergeall.py's -report and -auto (but not selective/interactive) usage modes.
For screenshots of this GUI in action, see the examples/Screenshots folder.

Usage: This is a .pyw -- it runs with no console on Windows.  Run this script
with no arguments, via icon clicks, command lines, or other.  Drag it out to
a desktop shortcut for quick access.  See folder docetc/launcher-configs for
a Windows desktop icon (attach it to a shortcut via right-click + Properties).

Uses widgets for choosing options and directory paths and viewing scrollable
mergeall stdout/stderr output, and supports saving and viewing mergeall's
output in log files.  Uses threads to stay active while waiting for mergeall
output lines.  Stays open to support multiple mergeall runs in one session --
report differences, run automatic updates, report on results, etc.  The
underlying mergeall.py was not changed: this is just a GUI shell.

This is easier than launch-mergeall-Console.py (no directory typing, and
quicker options selection), which is easier than running raw mergeall.py
command lines directly and manually from a console or shortcuts, though
mergeall.py command lines support more options (e.g., interactive mode).

As of version 1.4, this also uses threading to always remain responsive.
As of version 2.0, this also supports the mergeall auto-backups option.
As of version 3.0, this also supports mergeall's "-skipcruft" and Mac OS X.
See below for major changes made here, as well as open issues (TBDs).

--------------------------------------------------------------------------------

VERSION 3.1 CHANGES:
These were internals and command-line changes, and had no GUI impact.

VERSION 3.0 CHANGES:

This major release's changes were largely driven by a Mac OS X port,
and new usage on Linux.  Search for 'RunningOnMac' and '3.0' for Mac-ness.
Mergeall grew cruft file, symlink, and Windows long-path support in 3.0,
but most of these are not related to the GUI managed here.

App bar icon for Linux:
    Linux now gets a nice mergeall app bar (launcher) icon.  Mac icons
    remain a TBD, and Windows has always had window icons for this program.

Initial desktop logs folder for both Linux and Mac:
    The preset default for the logs folder now uses $HOME settings on these
    Unix platforms (Windows uses the user's desktop as before).

Redesign run verification dialog for Linux and Mac:
    The dialog popped up just before a run is launched was redesigned to
    make it more readable on Mac and Linux.  Its text was formerly fine
    on Windows, but fairly bunched-up elsewhere.  Also specialized the
    warning text to be less dire if backups are enabled.

Top-level window hack for Mac Tk:
    The Mac port required a top-level window hack to show buttons in Mac
    active-window style initially, using the recommended Tk 8.5 install.
    This is a complex story; see __main__ code in this file for details.

New toggle to suppress comparison lines, all platforms:
    The Mac port also inspired a new GUI toggle to suppress folder comparison
    message scrolling in the GUI (only).  This is on initially for Mac because
    the Mac Tk Text widget is VERY slow: ~30x slower than the mergeall process's
    output.  This toggle is off initially for Windows and Linux, because the GUI
    largely keeps up on these platforms, and the messages indicate progress.
    Still, the new toggle is available on these platforms too, because the GUI's
    scrolling can add a few seconds to mergeall runtime in some tests run on
    Windows, and may be a factor on slower machines; disable as desired.

    Either way, full details, including all comparison messages, are always
    available in saved log files popped up automatically at the end of a run.
    Note that this new toggle differs from 2.4's "-quiet" option and GUI toggle
    described below: "-quiet" disables output in mergeall itself before it ever
    reaches the GUI, because backup messages are arguably distracting in logs
    too; comparison messages are still useful in logs, if not the GUI.
    
    Unlike all other widgets (except the new post-run popup toggle), this toggle
    is also dynamic: setting it on/off while mergeall is running hides/shows 
    messages currently being generated.  The 'skipping' message is displayed 
    every time this toggle is turned back on, though it does not appear until 
    the next mergeall message is received.

    TBD: Mac text scrolls still seem painfully slow when there are many
    difference-report lines.  These could be also be suppressed, in addition
    to folder comparison messages, by: line.startswith(('comparing', '[', ' ')).
    This was ruled out, as it makes report-mode runs useless.  Text speed must
    be fixed by Tk's Mac developers (though one report on it went unreplied).

    UPDATE: see docetc/miscnotes/mac-weirdly-slow-tk85-text-scroll.py for a
    simple self-contained demo of the text scrolling slowness on ActiveState
    Tk 8.5.18 (the Tk recommended by python.org), Python 3.5, and OS X 10.11.
    It finishes scrolling in just 3-4 seconds on Windows, but 85 on Mac!
    
    UPDATE: recent heroic efforts at speeding Mac Text widget scrolling came
    up short.  It's simply slow in all Mac Tks tested - 8.5.9 through 8.5.18.
    The culprit seems to be update() calls required to redraw the widget after
    new text is inserted.   The code used to scroll uses normal techniques:
         statustxt.insert(END, line)    # add to Text widget
         statustxt.see(END+'-2l')       # reposition text
         statustxt.update()             # update display from within a loop
    1) Alternative scroll techniques tried had no effect on speed:
         statustxt.yview_scroll(2, 'units')
         statustxt.yview_moveto(1.0)
         statustxt.yview(END)
    2) Using update_idletasks() speeds scrolls only slightly, and all the GUI's
    controls are unresponsive while the text scrolls (an unacceptable effect):
         statustxt.update_idletasks()
    3) The after() timer loop's speed proved irrelevent, as the code mostly
    stays in the 'batch' inner loop, and doesn't rescedule after() events:
        statustxt.after(10, streamconsumer, linequeue, logfile, logpath)
    4) The update() call can be avoided by processing just 1 line per after()
    event (instead of the batch inner loop) with a very low delay count, but
    this has NO impact on scroll speed - implicit updates are also too slow:
        statustxt.after(1, streamconsumer, linequeue, logfile, logpath)
    5) This leaves disabling scrolls (the new toggle), or updating only after
    groups of lines are addded, which makes scrolling too jumpy and chaotic:
         global upcnt
         upcnt += 1
         if upcnt % 50 == 0:
             statustxt.update() 
    In the end, this is a Mac Tk bug, which hopefully is or will be addressed
    in later Tks on the Mac.  The new toggle is mergeall's best workaround.

New script to delete Mac ".*" cruft files:
    The Mac platform has a habit of creating lots of metadata files and a few
    folders, whose names all start with a ".", and which are sometimes treated
    as hidden.  These have meaning and purpose on a Mac, but are useless
    noise on other platforms.  For mergeall users on Windows and Linux whose
    archive might become infected with these files by an association with a
    Mac, a new script, "nuke-cruft-files.py," is provided to remove such files
    as a pre-step or post-step to mergeall runs that copy archives off a Mac
    See that script's extensive docstring for more details.

Disable widgets instead of erasing them, all platforms:
    Also partly for the Mac, the GUI was redesigned to enable/disable widgets
    as they fall in and out of relevance, instead of drawing/erasing them with
    pack() and pack_forget().  The latter scheme causes a noticeable flash on a
    Mac, but may have been too dramatic in general.  The "GO" button didn't
    cause flash, but enabling/disabling worked around an old text scroll issue.
    
New mode and toggle to skip system cruft files in both TO and FROM
    Inspired by the numerous ".*" files added to archives on Mac OS X, both
    mergeall and diffall grew a "-skipcruft" command line argument and mode,
    which skips known cruft files of all platforms in both FROM and TO.  The
    GUI also grew a new toggle to suport this new mode and argument. 

    When enabled, the net effect is that cruft files do not register as
    differences in report runs, and are not copied to, removed from, or replaced
    in the TO tree in update runs.  Such files thus stay on their generating
    platform only - they won't be transferred to intermediate drives and other
    computers, and won't be deleted from the generating platform by future
    merges.

    This option can be disabled, because users of a single platform may not
    care about their cruft, and some crufts may be more useful than others.
    Cruft files are defined in the mergeall_config.py file; users are
    encouraged to adjust the set of matching files as needed for their use.

Sanitize Unicode characters in message-scroll text outside Tk's BMP range
    Through 8.6, at least, Tk cannot display Unicode characers whose code
    points are outside range U+0000..U+FFFF (BMP, UCS-2).  Passing these to
    Tk raises an uncayght exception which leaves the GUI in an unpredictable
    state (usually half-drawn or hung).  To work around this, replace all
    such characters with the standard Unicode replacement character, which
    renders as a generic indicator.  This limit may be lifted in Tk 8.7.
    Update: also an issue for pathnames in Browse folder dialogs, fixed here.
    
More descriptive GUI labels and popups
    Toggle labels and ppups were given more explicit labels for clarity.

Allow editor popup to be disabled: config setting, new toggle
    The automatic popup of a text editor to view a saved mergeall log file
    can now be dsabled by a setting in mergeall_configs.py (on by default).
    The popup may seem overkill to users who require a view-only display.

    UPDATE: this is now switchable on/off by a new toggle in the GUI itself.
    The configs-file setting is retained, but used only for an intial value
    for the new toggle. The popup is normally unused, and GUI clicks are 
    easier than config edits.  This new toggle is dynamic, like the message
    scroll disabler; you can click it during a run to impact the popup. 

Configurable message text display area
    Users can now tailor the color, font, and initial size of the GUI's
    scrolled display area for mergeall messages, in mergeall_configs.py.
    The GUI's cosmetics were also polished in general along the way.

Mac changes to open dialogs (Browse for  folders TO, FROM, Logs)
    On Macs, use "message" ("title" is ignored), and add slide-down sheet
    style (versus popup window) via "parent=root" if configs file setting.
    Save dialogs seem to post titles correctly.  Mac's standard menu is
    customized here too, even though this program doesn't have one per se.

mergeall and cpall: copy unix symlinks, don't follow
    Avoid redundant data copies.  See mergeall.py and cpall.py for details.

Refocus on window after standard/common dialogs for Mac
    Run a root.focus_force() after all standard dialogs used here to force
    focus and active styling to be reset after dialog is closed.  Else, Mac
    users must click window after dialogs.  This may be AS Tk 8.5 only; has
    no effect on the initial style issue discussed earlier; and parent=root
    doesn't suffice to reset focus in the standard dialogs used here.

    TBD: should these dialogs also be made slide-down sheets on Mac like
    the open/save diaogs via parent=root, instead of modal popup windows?
    Some Mac purists may vote yea, but mergeall currently sides with variety,
    especially given that the mergeall GUI is a single-window interface.

mergeall output forced to ASCII in PyInstaller Windows exes
    A continuation of a long-running theme here, mergeall now forces stdout
    text to be ASCII when running as a frozen executable on Windows ONLY.
    See mergeall.py for details; this works around a likely PyInstaller bug.

--------------------------------------------------------------------------------

VERSION 2.4 CHANGES:
Added a toggle for the new "-quiet" script option that turns off per-file
"...backing up" log message printing in the underlying mergeall program.
These messages may make reading the log more work than it should be, and
don't add much information once backups are understood.  The new "-quiet"
toggle in the GUI is active only when backups are enabled.  Also tweaked
some GUI labels' text for clarity.  

VERSION 2.2 and 2.3 CHANGES:
These versions were optimizations and patches, and had no impact on this GUI.

VERSION 2.1 CHANGES:
This version's changes were command-line only, and had no effect on this GUI.

--------------------------------------------------------------------------------

VERSION 2.0 CHANGES:
1) Added Backups toggle and corresponding mergeall -backup command-line arg.
The new backups frame retains state but is shown/hidden when -auto is
selected/deselected, as it's applicable to -auto only here (not -report).
In the console launcher, -backup is applicable to both -auto and
selective (=[not -report]) modes.  Defaults to enabled in the GUI.

2) Moved log-file path chooser widgets down to log-file toggle button section.
The chooser retains state but is now shown/hidden when logging is toggled
on/off, as it's applicable when logging only.

3) Added "help" button that pops up the main usage doc in a web browser.
Packed in the run-mode frame's unused space.

4) The new "finished\n\n" message issued by mergeall solves the issue here
where the last output line was covered by repacking the GO button after resizes.
This works better, as it doesn't require the text display to scroll abruptly.

5) Use more descriptive text for the mode radio buttons (formerly was just
not "Report Only" and "Automatic Updates").

6) Error-check the FROM and TO paths here before trying to run mergeall, so
produces a GUI popup instead of mergeall text output message (log file path
was already being checked this way, because it required an open here).

7) Default to Desktop for log files on Windows (initial setting value only).

--------------------------------------------------------------------------------

VERSION 1.7.1 CHANGES:
None here (usage note, mergeall error message format change: see Revisions.html).

VERSION 1.7 CHANGES:
1) Minor bug fix for 2.X only -- add import of Tkinter's showerror when
using 2.X, else dialog never appears if bad log-file name.

2) Catch PermissionError (etc.) on log file open and report error in popup;
else fails silently on Windows, as ".pyw" has no console for exception text.

--------------------------------------------------------------------------------

VERSION 1.6 CHANGES: a new Python 2.X Unicode workaround, verify quits

Decoding an output line read from the spawned mergeall.py stream can fail
rarely in Python 2.X (only), if there are non-ASCII filename characters 
in the line.  This failure does not occur in 3.X.  Patched to catch the 
decode error and recover, with a notation at the front of the line in the 
GUI.  This change impacts text in the GUI display only; it does not impact
text in the log file, or the underlying mergeall process's run.  Also added
a verify dialog for the window close button to avoid accidental exits.

[Discussion] The Unicode decode failure seems a fundamental 3.X/2.X 
behavior difference.  After initial research, the most likely culprits 
appear to be:

a) The 2.X subprocess module mutates non-ASCII stream lines in transit.

b) The 2.X print statement generates either already-decoded text, or 
   wrongly-encoded bytes that do not respect the PYTHONIOENCODING setting.

In either case, this may reflect the fact that there is no notion of a 
truly binary bytes stream in 2.X -- the content of subprocess output is 
not as clearly defined as in 3.X's strict text/bytes dichotomy worldview.  

Note that setting PYTHONIOENCODING in the shell does not fix this failure.  
This environment variable is used for prints to the console, but this GUI
does only screen updates and log file writes itself, and exports this 
setting automatically to the mergeall subprocess for its prints.  Testing
verifies that os.environ settings are inherited by 2.X subprocesses too, 
even when spawned by the subprocess module.

Whatever the exact cause, this is a fundamental 2.X/3.X difference and 
another 2.X/3.X semantic incompatibility that goes well beyond syntax.  
In this case, adding an exception handler around the offending decode 
fixed the failure for all cases tested in 2.X, so this issue is closed.  
Given the other 2.X/3.X incompatibilities found and addressed in this 
project, though, it seems that writing dual-version 2.X+3.X code may be
a bit of a pipedream when it comes to realistic, practical programs.  
If in doubt, try running mergeall on Python 3.X instead of 2.X.

Also note that this patch applies only to the GUI launcher: PYTHONIOENCODING 
must still generally be set manually in your system shell when running either
the console launcher or script mergeall.py directly from a command line, if 
either may ever process and thus print non-ASCII filenames.  Else, such 
filenames may cause both scripts to abort, especially in Python 3.X.  The 
GUI launcher does not require this setting, as it automatically sets and 
propagates this variable to its mergeall.py subprocess, and never prints.

[3.0] UPDATE: for its PyIstaller frozen executable ONLY, mergeall must force
stdout text to ASCII, which isn't ideal, but is requied in this context to
avoid exceptions, and sidesteps this entire display-only issue.

--------------------------------------------------------------------------------

VERSION 1.5 CHANGES: Linux port and usage 

Pass shell=False on Linux/Unix (only) to subprocess.Popen, else starts a 
"python" interactive shell even though a full command sequence is passed. 
Linux users: see also release 1.5 notes in docs/Revisions.html for "#!"
pointers, and a possible NTFS timestamp skew issue for Windows/Linux
cross-platform syncs.

--------------------------------------------------------------------------------

VERSION 1.4 CHANGES: threads, streams, log files

1) Threading

   Add threading for the mergeall subprocess stream reader, a former TBD.
   This structure is more complex -- it trades a simple loop for two functions
   and multiple levels of loops -- but prevents the GUI from becoming blocked
   and unresponsive while waiting for a next line from the subprocess.

   This was normally not an issue: the GUI updates after each new line, and is
   used just for viewing after starting a mergeall run (and it's "GO" button is
   erased during this process to avoid overlapping runs).  However, blocked
   states are not natural in GUIs, and they can become apparent here if mergeall
   is busy copying large trees.  See book (PP4E) for more details and examples
   of GUI threading.

2) Stream Unicode decoding, take 2

   Force UTF8 encoding for prints in mergeall subprocess via PYTHONIOENCODING, 
   and use binary mode Popen stream reads + manual post-read UTF8 decoding here.
   Version 1.2 formerly made the subprocess's streams encoding match Popen's
   expectation (cp1252 on Windows) by using locale.getpreferredencoding(False);
   that works for reading the stream, but not for prints within mergeall itself.

3) 2.X log-file compatibility

   Use binary mode for the log files, writing the now-binary stream data.
   Former versions allowed for non-ASCII filenames in the log file text by
   using text mode and writing characters as UTF8, but the 2.X codecs.open()
   doesn't expand \n the same as 3.X's text-mode open() (see also next item).

4) 2.X unbuffered streams compatibility

   Temporarily dropped '-u' in mergeall spawn command-line, as it makes eolns
   (linebreak character sequences) \n in 2.X, but \r\n in 3.X.  This causes
   issues in log-file writes: without a \r\n on Windows, files are single lines.

   LATER UPDATE: the Python '-u' unbuffered flag has been reinstated.  Without
   it, mergeall output may not appear for 10 or more seconds on some machines
   and slower devices due to internal buffering.  Because this flag also makes
   line-breaks differ between Python 2.X and 3.X, though, also need to use
   special-case log-file writes to map all linebreaks to the platform's version.

   This is a complex 3.X/2.X compatibility issue, involving -u, Popen, and opens.
   Neither Popen text mode streams nor the 2.X codecs.open() will help.  The
   former can't be used because its internal encoding policy (per locale) is
   not broad enough to handle arbitrary Unicode filenames, and the latter always
   opens in binary mode, and so does no translations of linebreaks in the decoded
   text (3.X's text-mode open() does expand linebreaks by default).

   Could write binary lines in 2.X text-mode open() to expand \n on Windows,
   but that won't work in 3.X -- its text-mode open() expects Unicode strings,
   and always does encoding in addition to linebreaks.  Writing manually
   encoded text via open() in 3.X and codecs.open() in 2.X also won't work:
   2.X requires manual \r\n, which 3.X will by default expand to \r\r\n.
   3.X open() supports a 'newline' argument to turn off \n expansion, but
   this can't be used either, as it's not available in 2.X's codecs.open().
   As in, *punt*: write binary data with manual \n mapping for log files.

--------------------------------------------------------------------------------

TBD 1: No selective/interactive mode in GUI

mergeall's interactive mode is unavailable here as is, due to an outstanding
TBD regarding handling and interleaving of stdout for input() prompts in spawned
Python 3.X processes.  In practice, though, the -report and -auto modes have been
the only modes regularly used.  See launch-mergeall-Console.py for details on the
issue (the console launcher supports interactive mode, but without a log file).

TBD 2: 2.X compatibility for Unicode filenames?

This system works well on 3.X (it's main usage platform) and is largely 2.X
compatible, but the launchers may still have issues in some stream decoding
for non-ASCII filenames.  No such encoding exceptions occur on 2.X for the
raw mergall.py script, though some non-ASCII os.listdir results seem a bit
suspect in 2.X too.  More complete 2.X testing remains suggested exercise.
  --> Update: see also the 1.6 decoding change note above. <--

TBD 3: Decoupled versus single-process models?

The launchers currently use a standard decoupled model that spawns mergeall
and reads and decodes its streams.  There may be advantages to using a
single-process model that instead imports and calls mergeall's functions.
See docs\Lessons-Learned.html for more discussion on this alternative.

================================================================================
"""

#from __future__ import print_function   # 2.X compatibility: not needed here
APPNAME = 'mergeall'
VERSION = 3.1

# [3.0] for frozen app/exes, fix module+resource visibility (sys.path)
import fixfrozenpaths

import sys
if sys.version[0] >= '3':   # Py 3.X, but allow for Py 4.X too [3.0]
    import _thread, queue
    from tkinter import *
    from tkinter.messagebox   import askokcancel, showinfo, showerror
    from tkinter.filedialog   import Directory                 # saves last dir
    from tkinter.scrolledtext import ScrolledText
else:
    import thread as _thread, Queue as queue                   # Py 2.X compatibility
    from Tkinter              import *
    from tkMessageBox         import askokcancel, showinfo, showerror    # [1.7] 
    from tkFileDialog         import Directory
    from ScrolledText         import ScrolledText

    #import codecs
    #open = codecs.open   # [1.4] log binary mode from stream, not text files

# this script isn't too platform-specific, but avoid repeating this
RunningOnMac     = sys.platform.startswith('darwin')
RunningOnWindows = sys.platform.startswith('win')
RunningOnLinux   = sys.platform.startswith('linux')

import webbrowser, subprocess, time, os

# [3.0] data+scripts not in os.getcwd() if run from a cmdline elsewhere,
# and __file__ may not work if running as a frozen PyInstaller executable;
# use __file__ of this file for Mac apps, not module: it's in a zipfile;

MYDIR = fixfrozenpaths.fetchMyInstallDir(__file__)   # absolute

# [3.0] new doc, in this script's folder - but not necessarily '.' (cwd)
HELPFILE = os.path.join(MYDIR, 'UserGuide.html')

# [3.0] Mac OS X is pickier about file URLs
if RunningOnMac:
    HELPFILE = 'file:' + HELPFILE

# ANDROID [Apr1219] - open latest online version of user guide (all platforms should)
HELPURL = 'https://learning-python.com/mergeall-products/unzipped/UserGuide.html'

# [3.0] part of PP4E's guimaker module, copied here to avoid dependency
from guimaker_pp4e import fixAppleMenuBar

# [3.0] user configs: scrolled messages text area, log-file popup;
# for GUI settings, None = use Tk defaults: 24 lines high, 80 chars wide;
try:
    from mergeall_configs import (
        LOGEDITORPOPUP,
        DEFAULTLOGDIR,
        MACSLIDEDOWN,
        TEXTAREAHEIGHT, TEXTAREAWIDTH, TEXTAREAFONT, TEXTAREACOLOR,

        # ANDROID [Mar0819]
        DEFAULTFROMDIR, DEFAULTTODIR,                # entry prefills: see usage ahead
        BROWSELOGDIR, BROWSEFROMDIR, BROWSETODIR,    # chooser-dialog starts: ditto

        # ANDROID [Mar2819]
        LABELFONT,                                   # font if labels too wide to fit
        HEADERFONT)                                  # custom font for section headers

except Exception as why:
    # if any fail, all default (brutal, but simple)
    LOGEDITORPOPUP = True   # default: initial value for log-file popup toggle 
    DEFAULTLOGDIR  = None   # default: Desktop folder?, per running platform
    MACSLIDEDOWN   = False  # default: popup, not sheet, for folder dialogs
    TEXTAREAHEIGHT = 20     # initial number lines in message scroll widget
    TEXTAREAWIDTH  = None   # initial number characters per line, wrapped
    TEXTAREAFONT   = None   # scrolled messages font, None=Tk default font
    TEXTAREACOLOR  = None   # scrolled messages color(s), None=Tk default font

    # ANDROID [Mar0819]
    DEFAULTFROMDIR = None   # default: always Browse in GUI (entry prefill)
    DEFAULTTODIR   = None   # ditto
    BROWSELOGDIR   = None   # default: use tkinter start default (chooser dialog)
    BROWSEFROMDIR  = None   # ditto
    BROWSETODIR    = None   # ditto

    # ANDROID [Mar2819]
    LABELFONT      = None   # default to system default: best on most devices 
    HEADERFONT     = None   # default to small preset in code ahead

    print('Error in config file: %s' % why)   # to console, if any


def fixTkBMP(text):
    """
    [3.0] (copied from PyMailGUI) Tk <= 8.6 cannot display Unicode characters
    outside the U+0000..U+FFFF BMP (UCS-2) code-point range, and generates
    uncaught exceptions when tried (emojis kill programs!).  To address this,
    call this function to sanitize all text passed to the GUI for display.
    It replaces any non-BMP characters with the standard Unicode replacement
    character U+FFFD, which Tk displays as a highlighted question mark diamond.
    This workaround is coded to assume that Tk 8.7 will lift the BMP restriction,
    per a dev rumor.  It also assumes TkVersion has been imported from tkinter.
    Use here: filenames in mergeall messages scrolled text (rare, but true).
    """
    if TkVersion <= 8.6:
        text = ''.join((ch if ord(ch) <= 0xFFFF else '\uFFFD') for ch in text)
    return text


def isNonBMP(text):
    """
    [3.0] Return true if any char (codepoint) in text is outside Tk's BMP range.
    Used by folder dialogs to force initialfile=None when True for prior choice.
    """
    if TkVersion <= 8.6:
        return any(ord(ch) > 0xFFFF for ch in text)
    else:
        return False   # and assume Tk 8.7 will make this better...


def refocusWindow():
    """
    [3.0] Call after (most) standard dialogs to reset focus on the main
    window, else focus and active style are lost when dialog is closed.
    This may be a bug in ActiveState Tk 8.5 (TBD), but the fix is simple.
    """
    root.focus_force()   # else Mac requires a user click after dialogs



####################################################################################
# GUI BUILDER
####################################################################################



# font for headers in controls sections;
# could be user-configurable too, but seems arguably-better hardwired;
# [2.4] not just 'bold': if used as family name, Tk falls back on arial!
# [3.0] smaller font on Linux, else looks almost cartoonish;

HDRFONT = 'arial 14 bold'         # ('family', size, 'style? style?...')
if RunningOnLinux:
    HDRFONT = 'arial 12 bold'     # smaller is better on Linux (not Mac, Windows)   

# ANDROID - go smaller (arial==helvetica, which supports bold in font str or tuple);
# [Mar2819] also allow user to give header font in config file, along with label font;
#
HDRFONT = HEADERFONT or 'arial 6 bold'    # use small default if None


def makewidgets(root):
    """
    build the gui window, setup Browse/GO/other event handlers
    """
    # used on GO
    global dirents                                                    # folders
    global modevar, logvar, bkpvar                                    # settings
    global quietvar, cmpmsgsvar, cruftvar, logpopupvar                # more settings
    global gobutton, statustxt                                        # widgets


    #---------------------------------------------------------------------------
    # Event handlers (less ongobutton: ahead)
    #---------------------------------------------------------------------------

    # some use names in the enclosing func (which are actually globals)
    
    # one open dialog for entire run, saved in scope (closure) 
    opendirdlg = Directory() 

    def onbrowse(field, label):
        title = 'Choose mergeall %s folder' % label
        if RunningOnMac:                              # [3.0] specialize on Mac
            if MACSLIDEDOWN:
                # Mac: message (title ignored), slidedown window, custom text
                title = 'Choose mergeall %s folder' % label
                dlgkargs = dict(message=title, parent=root)
            else:
                # Mac: message (title ignored), popup window, standard text
                title = 'mergeall: Choose %s Folder' % label
                dlgkargs = dict(message=title)
        else:
            # Windows+Linux: the usual modal popup, standard text
            title = 'mergeall: Choose %s folder' % label
            dlgkargs = dict(title=title)

        # check prior pathname choice for emojis: kills dialog
        prior = opendirdlg.options.get('initialdir', '')
        if isNonBMP(prior):
            dlgkargs.update(dict(initialdir=None))   # for this call only

        # ANDROID [Mar0819] - use config starts if set because the Android tkinter
        # chooser GUI is tedious+slow (like its Linux cousin, but worse on phones);
        # Android uses different settings for entry prefills and dialogs starts;
        # 
        starts = dict(FROM=BROWSEFROMDIR, TO=BROWSETODIR, Logs=BROWSELOGDIR)
        if starts[label] and os.path.isdir(starts[label]):
            dlgkargs['initialdir'] = starts[label]
        else:
            pass                                   # use tkinter's default 
            # or: dlgkargs['initialdir'] ='/'      # dlgkargs.update() is overkill here
            # dlgkargs['initialdir'] = None        # this leaves None and kills isNonBMP

        chosendir = opendirdlg.show(**dlgkargs)
        if chosendir:
            field.delete('0', END)
            field.insert(END, fixTkBMP(chosendir))
        refocusWindow()   # [3.0] else requires click on Mac

    def onquit():
        answer = askokcancel('%s: Verify Exit' % APPNAME, 'Really quit mergeall now?')
        if answer:
            win.quit()              # win in enclosing scope; or sys.exit()
        else:
            refocusWindow()         # [3.0] else requires click on Mac

    def onmodetoggle():
        if modevar.get().startswith('REPORT'):          # hide/show backups frame [2.0]
            # [3.0] bkpfrm.pack_forget()                # -backup applies to -auto updates only
            bkpbtn.config(state=DISABLED)               # enabled/disable widgets [3.0]
            quietbtn.config(state=DISABLED)
            cruftbtn.config(text='Skip cruft items in FROM and TO?')   # ANDROID - shorter
                                 #'do not report as differences?')     # ANDROID
        else:
            # [3.0] bkpfrm.pack(expand=YES, fill=X)
            bkpbtn.config(state=NORMAL)
            quietbtn.config(state=NORMAL)
            cruftbtn.config(text='Skip cruft items in FROM and TO?')    # ANDROID - shorter
                                 #'do not copy, replace, or delete?')   # ANDROID

    def onlogtoggle():
        if logvar.get():                                # show/hide log-file path chooser [2.0]
            # [3.0] logdirfrm.pack(expand=YES, fill=X)  # chooser applies only if logging
            logbtn.config(state=NORMAL)
            logent.config(state=NORMAL)
            logpopupbtn.config(state=NORMAL)            # [3.0] ditto for log-popup toggle
            #logpopupbtn.config(state=DISABLED)         # ANDROID - webbrowser failed initially 
        else:
            # [3.0] logdirfrm.pack_forget()
            logbtn.config(state=DISABLED)
            logent.config(state=DISABLED)
            logpopupbtn.config(state=DISABLED) 

    def onbkptoggle():
        if bkpvar.get():                             # show/hide -quiet toggle button [2.4]
            # [3.0] quietbtn.pack(anchor=NW)         # -quiet applies only if doing backups,
            quietbtn.config(state=NORMAL)            # and whether saving to log file or not
        else:
            # [3.0] quietbtn.pack_forget()
            # quietvar.set(False)                    # keep prior setting, disabled=moot [3.0]
            quietbtn.config(state=DISABLED)

    def onhelp():
        # 
        # ANDROID [Apr1219]: webbrowser fails on Android (for reasons TBD), 
        # so spawn a shell command using the $BROWSER preset in Pydroid 3:
        # "am start --user 0 -a android.intent.action.VIEW -d %s"; Android
        # uses online version to pick up latest changes (others should too);
        #
        # ANDROID [Apr1919]: webbrowser _does_ work, but requires local file
        # URLs to start with "file://" and does not open a web browser for
        # local HTML files (they open in text editors); use the online URL
        # to ensure a web browser, and either os.system or webbrowser.open;
        #
        # ANDROID [Apr2119]: Pydroid 3 3.0 broke webbrowser and changed 
        # $BROWSER - use os.system() with a hardcoded command-line string;
        #
        brw = 'am start --user 0 -a android.intent.action.VIEW -d %s'
        cmd = brw % HELPURL
        os.system(cmd)          # was os.environ['BROWSER'], webbrowser.open(HELPURL)

        # other platforms code...
        """
        webbrowser.open(HELPFILE)                    # popup local file in web browser
        """


    #---------------------------------------------------------------------------
    # Build the GUI (link to handlers)
    #---------------------------------------------------------------------------


    #
    # MAIN WINDOW
    #
    
    win = root                                    # [3.0] allow for TopLevel() or Tk()
    win.title('mergeall %.1f' % VERSION)          # set main window title, [1.6] version
    win.protocol('WM_DELETE_WINDOW', onquit)      # [1.6] catch/verify window close

    # replace red (no, blue...) tk icon?
    iconfolder = os.path.join(MYDIR, 'icons')
    try:
        if RunningOnWindows:
            # try Windows window icon
            icnpath = os.path.join(iconfolder, 'mergeall.ico')
            win.iconbitmap(icnpath)

        elif RunningOnLinux:
            # try Linux app-bar icon [3.0]
            icnpath = os.path.join(iconfolder, 'mergeall.gif')
            imgobj = PhotoImage(file=icnpath)
            win.iconphoto(True, imgobj)

        elif RunningOnMac or True:
            # Mac OS X: neither of the above works (yet?) [3.0]
            # Macs require apps for most icon contexts
            raise NotImplementedError
        
    except Exception as why:
        # punt: bad file/call or platform (Mac OS X TBD)
        pass

    # [3.0] on Mac, customize app-wide automatic top-of-display menu
    fixAppleMenuBar(root, 'mergeall',
                    helpaction=onhelp, aboutaction=None, quitaction=onquit)

    ctrlfrm = Frame(win)
    ctrlfrm.pack(expand=NO, fill=BOTH, side=TOP)

    
    #
    # MAIN TO/FROM DIRECTORY CHOOSERS
    #
    
    dirsfrm = Frame(ctrlfrm, relief=GROOVE, border=3)
    dirsfrm.pack(fill=X)
    Label(dirsfrm, text='Main Folders', font=HDRFONT).pack()   # [2.4] font

    rowsfrm = Frame(dirsfrm)
    rowsfrm.pack(expand=YES, fill=X)
    rowsfrm.columnconfigure(1, weight=1)

    dirents = {}
    for (row, key) in enumerate(('FROM', 'TO')):
        rowsfrm.rowconfigure(row, weight=1)
        Label(rowsfrm, 
              font=LABELFONT,                 # ANDROID - configurable [Mar2819]
              text=key + ' folder:').grid(row=row, column=0, sticky=E)

        dirent = Entry(rowsfrm)
        dirent.insert(END, 'enter or browse...')
        dirent.grid(row=row, column=1, sticky=EW)

        handler = lambda dirent=dirent, key=key: onbrowse(dirent, key)   # current!
        Button(rowsfrm, text='Browse...',
               command=handler).grid(row=row, column=2)
        dirents[key] = dirent

    # ANDROID [Mar0819] - prefill to avoid tedious Android/phone chooser dialog
    for (key, prefill) in [('FROM', DEFAULTFROMDIR), ('TO', DEFAULTTODIR)]:
        if prefill and os.path.isdir(prefill):
            dirents[key].delete('0', END)
            dirents[key].insert(END, prefill)


    #
    # RUN MODE RADIO BUTTONS: -report or -auto
    #
    
    radiofrm = Frame(ctrlfrm, relief=GROOVE, border=3)
    radiofrm.pack(fill=X)
    Label(radiofrm, text='Run Mode', font=HDRFONT).pack()

    # help button: use empty space in run mode frame [2.0]
    # ANDROID [Apr1219]: use default color to avoid loss, was [relief=FLAT, bg='white')]
    #
    helpbtn = Button(radiofrm, text='Help', command=onhelp)
    helpbtn.pack(side=RIGHT, anchor=NE)
    #helpbtn.config(state=DISABLED)                 # ANDROID - webbrowser failed initially
 
    modevar = StringVar()
    modes = ['REPORT: show differences only',       # ANDROID - shorter
             'UPDATE: make TO the same as FROM']    # ANDROID - shorter
    for mode in modes:
        Radiobutton(radiofrm,
                    text=mode,
                    font=LABELFONT,                 # ANDROID - configurable [Mar2819]
                    variable=modevar,
                    value=mode,
                    command=onmodetoggle).pack(side=TOP, anchor=NW)
    modevar.set(modes[0])

    # [3.0] skip cruft files in FROM and TO - don't copy/remove/replace or report 
    cruftvar = BooleanVar()                                
    cruftvar.set(True)
    cruftbtn = Checkbutton(radiofrm, 
                           text='Skip cruft items in FROM and TO?',   # ANDROID - shorter
                                #'do not report as differences?',     # ANDROID
                           font=LABELFONT,                            # ANDROID - configurable [Mar2819]
                           variable=cruftvar)
    cruftbtn.pack(side=LEFT)


    #
    # MESSAGE TOGGLES + LOG DIRECTORY CHOOSER
    #
    
    msgfrm = Frame(ctrlfrm, relief=GROOVE, border=3)
    msgfrm.pack(fill=X)
    Label(msgfrm, text='Messages', font=HDRFONT).pack()

    logfrm = Frame(msgfrm)
    logfrm.pack(side=TOP, fill=X)
    logvar = BooleanVar()                                  # Intvar works too
    logvar.set(True)                                       # else default=off (eibti)
    Checkbutton(logfrm,
                text='Save logfile to folder? ',           # ANDROID - shorter
                font=LABELFONT,                            # ANDROID - configurable [Mar2819]
                variable=logvar,
                command=onlogtoggle).pack(side=LEFT)

    # log-file dir chooser by toggle, unhide/hide when toggled on/off [2.0]
    key = 'Logs'
    logdirfrm = Frame(logfrm)
    logdirfrm.pack(expand=YES, fill=X)
    handler = lambda: onbrowse(logent, key)     # last, not current (!) [see above]
    logbtn = Button(logdirfrm, text='Browse...', command=handler)
    logbtn.pack(side=RIGHT)

    logent = Entry(logdirfrm)
    logent.insert(END, 'enter or browse...')    # ANDROID - shorter
    logent.pack(side=LEFT, expand=YES, fill=X)
    dirents[key] = logent
    
    # [3.0] logdirfrm.pack_forget()   # till logs toggled on (now enabled/disabled)
    logbtn.config(state=NORMAL)       # till logs toggled on (initially are)
    logent.config(state=NORMAL)

    # fill initial/default logs folder value  
    if DEFAULTLOGDIR and os.path.exists(DEFAULTLOGDIR):
        # [3.0] allow user-configs file to give initial default 
        defaultlogs = DEFAULTLOGDIR
    else:
        try:
            # [2.0] try user's Desktop on Windows (has HOMEPATH but no HOME)
            defaultlogs = r'C:\Users\%s\Desktop' % os.environ['username']
            assert os.path.exists(defaultlogs)
        except:
            try:
                # [3.0] try same on Linux and Mac OS X (TBD: or Documents?)
                defaultlogs = os.path.join(os.environ['HOME'], 'Desktop')
                assert os.path.exists(defaultlogs)
            except:
                # ANDROID - try internal-storage docs folder (on some phones);
                # [Mar0819] don't use /sdcard/Documents as a prefill (made by 
                # MS Office apps); create the preset if it doesn't exist; but
                # still use DEFAULTLOGDIR as the prefill instead, if it's set;
                #
                try:
                    defaultlogs = '/sdcard/Admin-Mergeall'   # was '/sdcard/Documents'
                    if not os.path.exists(defaultlogs):      # make now if needed
                        os.mkdir(defaultlogs)
                    assert os.path.isdir(defaultlogs)        # exists + is a folder
                except:
                    defaultlogs = None

    if defaultlogs:
        # may be unset or fail on some Unix and Windows =>  empty, Browse
        logent.delete('0', END)
        logent.insert(END, defaultlogs)

    # [3.0] allow comparison messages to be suppressed (mostly for Mac speed);
    # this toggle (only) is dynamic: can change effect while output scrolling; 
    cmpmsgsvar = BooleanVar()
    if RunningOnMac:
        cmpmsgsvar.set(True)               # initial=suppress on Mac OS X
    else:
        cmpmsgsvar.set(False)              # off on Windws/Linux: fast GUI
    Checkbutton(msgfrm,
                text='Hide comparison messages?',     # ANDROID - shorter [Mar2819]
                font=LABELFONT,                       # ANDROID - configurable [Mar2819]
                variable=cmpmsgsvar).pack(side=LEFT, anchor=NW)

    # [3.0] allow log-file popup editor to be suppressed in the GUI per run;
    # the configs-file entry added previously now gives an initial value only;
    logpopupvar = BooleanVar()
    logpopupvar.set(LOGEDITORPOPUP)
    logpopupbtn = Checkbutton(msgfrm,
                text='Popup logfile?',         # ANDROID - shorter [Mar2819]
                font=LABELFONT,                # ANDROID - configurable [Mar2819]
                variable=logpopupvar)
    logpopupbtn.pack(side=RIGHT, anchor=NE)
    #logpopupvar.set(False)                    # ANDROID - webbrowser failed initially 
    #logpopupbtn.config(state=DISABLED)        # ANDROID - webbrowser failed initially
    

    #
    # BACKUP TOGGLES
    #
    
    # enable/disable when '-auto' run mode selected/deselected [2.0] [3.0]
    bkpfrm = Frame(ctrlfrm, relief=GROOVE, border=3)
    bkpfrm.pack(fill=X)
    Label(bkpfrm, text='Backups', font=HDRFONT).pack()
    
    bkpvar = BooleanVar()
    bkpbtn = Checkbutton(bkpfrm,
                text='Save TO items replaced or deleted, note adds?',   # ANDROID - shorter [Mar2819]
                font=LABELFONT,                                         # ANDROID - configurable [Mar2819]
                variable=bkpvar,
                command=onbkptoggle)
    bkpbtn.pack(anchor=NW)
    bkpvar.set(True)                 # initial=on: do backups

    # [3.0] bkpfrm.pack_forget()     # till -auto selected
    bkpbtn.config(state=DISABLED)    # till -auto selected
    
    # [2.4] support -quiet mode: omit "...backing up" per-file log messages
    quietvar = BooleanVar() 
    quietbtn = Checkbutton(bkpfrm,
                           text='Disable per-item backup messages?',    # ANDROID - shorter [Mar2819]
                           font=LABELFONT,                              # ANDROID - configurable [Mar2819]
                           variable=quietvar)
    quietbtn.pack(anchor=NW)
    quietvar.set(True)                # init=suppress (though good for errors+feedback)
    
    # [3.0] quietbtn.pack_forget()    # only if backups selected, but will be initially
    quietbtn.config(state=DISABLED)   # till -auto AND -backups selected  
  

    #
    # MESSAGES SCROLLED TEXT + 'GO' BUTTON
    #
    
    # 'go' always hidden during run to prevent overlapping run launches
    statustxt = ScrolledText(win)
    statustxt.config(state=DISABLED)  # ANDROID [Mar2319]: readonly text to avoid keyboard
    
    # [3.0] user configs: size, font, color (no border - color sets off better)
    try:
        if TEXTAREAHEIGHT: statustxt.config(height=TEXTAREAHEIGHT)
        if TEXTAREAWIDTH:  statustxt.config(width=TEXTAREAWIDTH)
        if TEXTAREAFONT:   statustxt.config(font=TEXTAREAFONT)
        if TEXTAREACOLOR:
            if isinstance(TEXTAREACOLOR, str):
                statustxt.config(bg=TEXTAREACOLOR)
            elif isinstance(TEXTAREACOLOR, tuple):
                statustxt.config(bg=TEXTAREACOLOR[0])
                statustxt.config(fg=TEXTAREACOLOR[1])
            else:
                print('Bad color value in config file')
    except Exception as why:
        print('Bad config setting: %s' % why)
    
    gobutton = Button(win, text='GO: run mergeall', font=HDRFONT,
                      command=ongobutton)
    gobutton.pack(side=BOTTOM)
    statustxt.pack(side=TOP, expand=YES, fill=BOTH)    # pack last=clip first



####################################################################################
# ON "GO": SPAWN MERGEALL PROCESS
####################################################################################



# [1.4] how spawned mergeall subprocess's text is written and decoded here
STREAM_ENCODE = 'utf8'

EOF_SENTINEL = []            # stream lines read will never be a list
linequeue = queue.Queue()    # infinite-size shared queue of objects



def ongobutton():
    """
    on GO button press: fetch gui values, confirm run, launch mergeall
    """
    # set in makewidgets
    global dirents                                         # folders
    global modevar, logvar, bkpvar, quietvar, cruftvar     # settings
    global gobutton, statustxt                             # widgets

    # [3.0] for comparison message suppresssion
    global firstcompareline
    firstcompareline = True               # reset before each run
    
    # get inputs from GUI
    fromdir = dirents['FROM'].get()       # directory fields
    todir   = dirents['TO'].get()
    logdir  = dirents['Logs'].get()
    mode    = modevar.get()               # runmode radiobtn
    dolog   = logvar.get()                # logfile checkbtn
    dobkp   = bkpvar.get()                # backups checkbtn
    quiet   = quietvar.get()              # quiet mode checkbtn [2.4]
    docruft = cruftvar.get()              # skip cruft files [3.0]
    
    # config run    
    modearg = '-report' if mode.startswith('REPORT') else '-auto'
    if dobkp and modearg != '-report':
        modearg += ' -backup'             # [2.0] backup replacements/removals
        if quiet:
            modearg += ' -quiet'          # [2.4] no per-file backup log messages

    if docruft:
        modearg += ' -skipcruft'          # [3.0] ignore cruft files in FROM and TO

    if not dolog:
        logpath = logfile = None
    else:
        datestamp = time.strftime('date%y%m%d-time%H%M%S')
        logpath   = logdir + os.sep + 'mergeall-%s.txt' % (datestamp)

    # confirm run
    if modearg.startswith('-report'):
        runtype = 'REPORT-ONLY RUN'
        warning = 'This run will not change your data.'
    else:
        runtype = 'AUTO-UPDATE RUN'
        warningbase = (
            # ANDROID [Mar2319] - manually line break for fit (but Tk still truncates paths)
            #
            # '*WARNING*: by design, this may change your TO folder tree in-place, '
            # 'by adding, replacing, and deleting files and folders as needed '
            # 'to make TO the same as FROM.')
            #
            '*CAUTION*: by design, this run may change\n'
            'your TO folder tree in-place, by adding,\n'
            'replacing, and deleting files and folders\n'
            'as needed to make TO the same as FROM.')

        warningmore = (
            # ANDROID [Mar2319] - manually line break for fit (but Tk still truncates paths)
            #
            # '  Because backups are disabled, any such changes '
            # 'will be permanent and irrevocable.')
            #
            '\nBecause backups are disabled, any such\n'
            'changes cannot be undone.')
        warning = warningbase + ('' if dobkp else warningmore)
        
    confirm = askokcancel('%s: Confirm Run' % APPNAME,
            'About to run:\n'
            'mergeall.py %s\n\n'
            'FROM:\n%s\n\n'
            'TO:  \n%s\n\n'
            'Logging output to:\n%s\n\n'
            '%s\n\n'
            'Start this %s?' %
                          (modearg, fromdir, todir, logpath, warning, runtype))

    notruntitle = '%s: Not Run' % APPNAME
    if not confirm:
        showinfo(notruntitle,  'The mergeall run was cancelled.')
    elif not os.path.exists(fromdir):
        showerror(notruntitle, 'Please select a valid mergeall FROM folder.')      # [2.0] popup
    elif not os.path.exists(todir):
        showerror(notruntitle, 'Please select a valid mergeall TO folder.')        # [2.0] popup
    elif dolog and not os.path.exists(logdir):
        showerror(notruntitle, 'Please select a valid mergeall log file folder.')  # or sooner?
    else:
        # [1.4] log uses binary mode for now-binary data from stream
        if dolog:
            try:
                logfile = open(logpath, 'wb')
            except:
                # [1.7] catch PermissionError and show popup (else silent for .pyw)
                showerror(notruntitle, 'Please select a writeable log file folder.')
                return

        # proceed with mergeall
        # [3.0] gobutton.pack_forget()     # hide/erase button
        gobutton.config(state=DISABLED)    # keep but disable
        statustxt.config(state=NORMAL)     # ANDROID [Mar2319]: enable text for changes during run
        statustxt.delete('1.0', END)       # clear last run text
        

        # [1.4] force UTF8 prints in mergeall, use binary streams + manual decode here;
        # this setting is inherited by the spawned mergeall subprocess for its prints;
        os.environ['PYTHONIOENCODING'] = STREAM_ENCODE

        # config mergeall command (sequences: auto-quoted by subprocess)
        extras = {}
        if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
            # pyinstaller exe [3.0]
            # run frozen executable directly, not script through python
            
            freezename = 'mergeall.exe' if RunningOnWindows else 'mergeall'
            mergeallpath = os.path.join(MYDIR, freezename)
            os.environ['PYTHONUNBUFFERED'] = 'True'           # -u equiv (iff env?)
            cmdseq = [mergeallpath,                           # frozen executable
                      fromdir,                                # '/' ok on Windows
                      todir] + modearg.split()                # 'a b?' -> ['a', 'b'?]
            
            if RunningOnWindows:
                # else spawn hangs unless launcher uses --console (with popup!)
                # startupinfo, env=os.environ, and creationflags are irrelevant
                extras.update(stdin=subprocess.DEVNULL)

        else:
            # py2app Mac app or source (original code)
            # relative script path works whether run here or via desktop shortcut
            # [3.0] but not in os.getcwd() if run from a cmdline elsewhere

            # ANDROID - hardcode sys.executable, else empty in Pydroid 3: kills Popen
            #
            # ANDROID [Apr1919]: Pydroid 3's 3.0 release moved its Python from the
            # first of the following paths to the second, breaking this workaround:
            #    /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python
            #    /data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/bin/python
            # to allow for both paths--and be platform agnostic in general--read the
            # result of a 'which python' shell command instead of using literal strs;
            #
            sys.executable = os.popen('which python').read().rstrip()  # path to Python exe

            scriptname = 'mergeall.py'
            mergeallpath = os.path.join(MYDIR, scriptname)      
            cmdseq = [sys.executable, '-u',                   # [1.4] need -u unbufferred
                      mergeallpath,                           # script file (app or source)
                      fromdir,                                # '/' ok on Windows
                      todir] + modearg.split()                # 'a b?' -> ['a', 'b'?]

        # [1.5] shell should be True on Windows so that it uses filename associations,
        # but False on Linux so that it doesn't just start a "python" interactive shell
        doshell = RunningOnWindows
        
        # spawn mergeall
        # use subprocess: os.popen/spawnv not enough, popen2 is 2.X only;
        subproc = subprocess.Popen(
                      cmdseq,                     # a string cmd may fail on Unix
                      shell=doshell,              # [1.5] see note above, platform specific
                      universal_newlines=False,   # [1.4] binary mode, manual decode/eoln
                      stdout=subprocess.PIPE,     # capture sub's stdout here
                      stderr=subprocess.STDOUT,   # route sub's stderr to its stdout
                      **extras)

        # read and process mergeall's output: reader thread + timer-based poller
        _thread.start_new_thread(streamreader, (subproc.stdout, linequeue))
        streamconsumer(linequeue, logfile, logpath)

        # returns here immediately: a thread and timer-event loop are now running

    # [3.0] for all cases, else requires click on Mac
    refocusWindow()



####################################################################################
# MERGEALL PROCESS HANDLER: THREAD + POLLER
####################################################################################



def streamreader(stream, linequeue):
    """
    [1.4] In a parallel thread - read the mergeall subprocess's stdout/stderr
    stream, and post its lines to a queue for the GUI to read and display
    on timer event callbacks; this way, the GUI isn't blocked/paused during
    long-running copies or other actions in the spawned mergeall script;
    """
    for line in stream:             # read stdout+stderr lines: may block this thread
        linequeue.put(line)         # place on queue to be picked up by GUI thread
    linequeue.put(EOF_SENTINEL)     # write sentinel at eof and exit: subprocess ended



def streamconsumer(linequeue, logfile, logpath):
    """
    [1.4] In the main GUI thread - run a timer-based loop to poll for, read,
    and display and log stream lines from the shared thread queue until the
    reader thread sends the end-signal sentinel value on the queue.  The main
    GUI thread running this code thus remains active between mergeall output
    lines.  A nested loop is also used here to process lines in batches so the
    GUI's response to lines is quick, but it calls update() to remain active.
    
    At this point we have 2 threads and a process (connected by queue and stream)
    and 3 or 4 loops going at once -- the main GUI thread runs a timer event loop
    to poll the queue, and runs the nested line-batch loop here; the spawned
    stream-reader thread runs a blockable reading loop and posts lines to the
    queue; and the spawned mergeall process runs its own file-processing loops
    to create output lines eventually displayed in the GUI here.
    """
    global statustxt, gobutton           # widgets
    global firstcompareline              # state 
    global cmpmsgsvar, logpopupvar       # settings


    def trydecode(binline, stream_encode):
        """
        [1.6] line decode can fail in Python 2.X due to a TBD library
        incompatibility issue; see 1.6 note near top of this script;

        if uncaught, GUI is dead but unclosed, and the error message does
        not appear in GUI -- because the error occurs here in GUI instead 
        of subproc, its text goes to console here (if any), not to subproc 
        stream queue, and the subproc is apparently terminated in 2.X;

        also note that "line" text here is used for the GUI display only:
        it doesn't show up in the binary log file, and this has no impact 
        on the underlying mergeall process, which proceeds unaffected;

        [3.0] changed the error message to be a str instead of a bytes;
        the latter would surely fail later in the caller under python 3.X,
        but the decode here probably only ever failed on python 2.X, where
        a bytes result works because bytes is really just a synonym for str;
        """
        try:
            line = binline.decode(stream_encode)        # [1.4] manual decode here, match subproc
        except UnicodeDecodeError:
           #line = b'(UNDECODABLE LINE): ' + binline    # [1.6] don't let this kill the GUI in 2.X
            line = '(UNDECODABLE LINE: see log file)\n' # [3.0] use str, but drop the content
        return line


    try:
        binline = linequeue.get(block=False)       # check the queue
    except queue.Empty:
        pass                                       # nothing posted yet: reschedule and wait
    else:
        # process a batch of 1 or more objects
        while True:
            
            if binline != EOF_SENTINEL:                
                # process the next line string: GUI + logfile
                
                eoln = os.linesep                         # local line-end: \r\n Windows, \n Unix
                line = trydecode(binline, STREAM_ENCODE)  # [1.4] manual decode here, match subproc [1.6] 
                line = line.replace(eoln, '\n')           # [1.4] and fix any Windows eolns for tk

                # [3.0] sanitize Unicode in line to be displayed in GUI
                line = fixTkBMP(line)
                
                # [3.0] allow comparison messages to be suppressed in the GUI, dynamically
                anycompare = ('comparing', '"comparing', "'comparing")   # ascii() in Windows exe!
                if line.startswith(anycompare):                          # in line content?
                    if cmpmsgsvar.get():                                 # suppress toggle on?
                        if firstcompareline:
                            # show first line only for top-level dirs, plus message
                            firstcompareline = False
                            statustxt.insert(END, line)
                            statustxt.insert(END, 'Folder comparison messages '
                                                  'are being suppressed in the GUI...\n')
                            statustxt.see(END+'-2l')      # scroll to new end of text
                        else:
                            # skip all other compare lines in GUI (only)
                            pass
                    else:
                        # show comparison line, reset to show message if toggled on+off
                        firstcompareline = True           # reshow msg if toggled again
                        statustxt.insert(END, line)       # add to end of text widget
                        statustxt.see(END+'-2l')          # scroll to new end of text
                else:
                    # show all other lines normally, don't reset for message 
                    statustxt.insert(END, line)           # add to end of text widget
                    statustxt.see(END+'-2l')              # scroll to new end of text
                                                          # '-2l' = before empty auto \n at end
                                                          
                # force GUI to show/respond now (else dead during batch)
                statustxt.update() 

                # write binary stream line to binary logfile
                if logfile:                                    # also save to log file? [1.4]: binary
                    eoln = os.linesep.encode()                 # must be bytes in 3.X (no-op in 2.X)
                    binline = binline.replace(b'\r\n', b'\n')  # [1.4] got just \n from '-u' in 2.X only
                    binline = binline.replace(b'\n', eoln)     # replaces are no-op in 3.X and unix 
                    logfile.write(binline)

                # read next line if any, else goto reschedule and wait
                if linequeue.empty():
                    break
                else:
                    binline = linequeue.get(block=False)   # back to top of loop
                
            else:
                # reader thread posted eof sentinel and exited: close out the run
                showinfo('%s: Finished' % APPNAME,
                    'The mergeall run has finished.' +
                    ('' if not logfile else
                       ('\nSee its log file in the popup window.' if logpopupvar.get() else
                        '\nSee its log file in the logs folder.')))
                refocusWindow()   # [3.0] else requires click on Mac
            
                # if logging: close logfile, show in editor if toggled on [3.0]
                if logfile:
                    logfile.close()                  
                    if logpopupvar.get():
                        # [3.0] Mac OS X is pickier about file URLs
                        if RunningOnMac:
                            logpath = 'file:' + logpath
 
                        #
                        # ANDROID [Apr1219] - webbrowser fails on Android (for reasons TBD), 
                        # so spawn a shell command using the $BROWSER preset in Pydroid 3:
                        # "am start --user 0 -a android.intent.action.VIEW -d %s";
                        # safe to assume logpath is accessible (else can't write anyhow);
                        #
                        # ANDROID [Apr1919] - webbrowser _does_ work, but requires local file
                        # URLs to start with "file://"; prefix as required, and use either 
                        # os.system or webbrowser.open to display the logfile in the GUI;
                        #
                        # ANDROID [Apr2119]: Pydroid 3 3.0 broke webbrowser and changed 
                        # $BROWSER to skip "file://" - use os.system() +  hardcoded command;
                        #
                        brw = 'am start --user 0 -a android.intent.action.VIEW -d %s'
                        url = 'file://' + logpath
                        cmd = brw % url
                        os.system(cmd)          # _not_ os.environ['BROWSER'], webbrowser.open()

                        # other platforms code...
                        """
                        # webbrowser opens text files in Notepad on Windows,
                        # gedit on Linux, and TextEdit on Mac OS X (but YMMV)
                        webbrowser.open(logpath)      # assume never raises exc
                        """

                # [3.0] reenable new runs now
                gobutton.config(state=NORMAL)
                statustxt.config(state=DISABLED)    # ANDROID [Mar2319]: back to readonly to avoid keyboard

                # [3.0] the following had odd text scrolls after vertical resizes
                # statustxt.pack_forget()
                # gobutton.pack(side=BOTTOM)                        # unhide button
                # statustxt.pack(side=TOP, expand=YES, fill=BOTH)   # pack last=clip first
 
                # exit the timer events loop: back to waiting on user
                return

        # end batch while loop
        
    # reschedule and wait: check queue 10 times per second (msecs)
    statustxt.after(100, streamconsumer, linequeue, logfile, logpath)



####################################################################################
# MAIN/TOP-LEVEL LOGIC
####################################################################################



if __name__ == '__main__':

    if not RunningOnMac:                     # Windows and Linux: normal
        root = Tk()                          # 'root' is used elsewhere
        makewidgets(root)
        root.mainloop()
       
    else:
        
        #----------------------------------------------------------------------------
        # [3.0] Mac OS X hack: a partial workaround for a bug in the recommended AS
        # Mac Tk 8.5 for Python 3.5.  Hide/unhide main window so it shows its radio
        # and check buttons in Aqua's active-window style (default blue) immediately.
        # This code fixes style at initial opening only; style can be lost on popups
        # and minimize/restore -- click this window, and possibly others first, to
        # reset style as needed.  More: docetc/miscnotes/mac-main-hack-notes.txt.
        #
        # This may be fixed in Tk 8.6, which may be supported by py.org's Python 3.X
        # someday, and is supported by homebrew's Python distribution (to be tested).
        # Unlike frigcal and pymailgui, a lift() is not enough here, whether clicked
        # to open in the mac python launcher, or run from a 'python3' command line.
        #
        # Update: root.force_focus() is now used to restore window active state after
        # dialogs, but doesn't help for deiconifies caught via <Map> or <Visibility>
        # events, and has no impact here on initial state in all codings attempted.
        # 
        # Update: the following don't help even if they do a focus_force (why?):
        # root.bind('<Map>', onUnhide) + root.after(2000, refocusWindow),
        # root.createcommand('::tk::mac::onShow', onShow),
        # root.createcommand('::tk::mac::ReopenApplication', onReopen)
        #
        # UPDATE: losing focus in deiconifies in AS Tk 8.5 was finally fixed by the
        # hideous workaround below, which creates and then immediately destroys a 
        # new but lowered (and hence invisible) top-level window on the Mac 
        # reopen-app even (i.e., Doc and app icon clicks).  The widgets flash off
        # and on momentarily (and the temp window may flash if mergeall is not at
        # fullscreen), but otherwise are active styled.  Tk 8.6 status tbd... 
        #----------------------------------------------------------------------------

        root = Tk()
        makewidgets(root)

        # fix tk focus loss on startup
        root.withdraw()
        root.lift()
        root.after_idle(root.deiconify)
        
        # fix tk focus loss on deiconify
        def onReopen():
            #print(root.state())    # always normal
            root.lift()
            root.update()
            temp = Toplevel()
            temp.lower()
            temp.destroy()
        root.createcommand('::tk::mac::ReopenApplication', onReopen)

        root.mainloop()
        

        """
        # an alternative open workaround: a bogus Tk root, iconified after 2 seconds:
        root = Tk()
        root.protocol('WM_DELEsTE_WINDOW', lambda: None)
        Label(root, text='Welcome to mergeall', width=25, height=5).pack()
        makewidgets(Toplevel())
        root.after(2000, root.iconify)
        root.mainloop()
        """

        """
        # a 15-line AppleScript alternative omitted here for space (and humanity...)
        """



[Home] Books Programs Blog Python Author Training Search Email ©M.Lutz