![[Home page]](../PythonPowered.gif "Home page"){kind=small}
| Books | Code | Blog | Python | Author | Train | Find | ©M.Lutz |
File: mergeall-android-scripts/_readme-items/PYDROID3-TKINTER/CODE/launch-mergeall-GUI.pyw
#!/usr/bin/python
"""
================================================================================
launch-mergeall-GUI.pyw:
desktop GUI launcher (part of the mergeall system)
-Copyright and author: © M.Lutz, 2014-2019 (http://learning-python.com)
-License: provided freely, but with no warranties of any kind
#-------------------------------------------------------------------------------
# ANDROID VERSION, January-April 2019
# Per http://learning-python.com/mergeall-android-scripts/_README.html#toc8.
#
# Replace original file with this custom version on your phone (only).
# See "# ANDROID" for all changes applied here and in mergeall_configs.py.
# These changes are for Android only, do not impact the command-lines mode
# of Mergeall, and might be merged into the original file in a later release.
#
# Recent changes (search here for their [date] labels to find changed code):
#
# [Apr2119] Pydroid 3 3.0 broke webbrowser: use os.system(cmd) with a
# hardcoded Android activity-manager command line instead
# (3.0's $DISPLAY breaks module, $BROWSER kills "file://").
#
# [Apr1919] Fix merge-spawn workaround to the Pydroid 3 empty sys.executable
# bug, to accommodate the different Python path in Pydroid 3's 3.0.
# The fix reads a spawned 'which python' command to be path agnostic.
# Also go back to using webbrowser for help and logfiles: it does
# work, but iff files use '"file://" and HTML docs use online URLs;
# see _openbrowser.py for a demo and more details.
#
# [Apr1219] Reenable help and popup-logfile buttons, and fix them to open with
# an os.system() spawn of an activity-manager command instead of Py's
# webbrowser; open online version of help, for its latest changes.
# Also use default color for "Help" - tkinter loses a custom bg.
#
# [Mar2819] Shorten label text again and further, for smaller phone displays.
# Labels now fit >= 5.5" screens; if still too wide, set the new
# LABELFONT user config, or search for "# ANDROID - shorter" to edit.
# Also for smaller displays, resplit run-dialog text (but not paths).
#
# [Mar2319] Made message text readonly to avoid keyboard popups on slow swipes.
# This precludes copy/paste, but the on-screen keyboard is annoying.
# Also manually split run-dialog text (but Tk still truncates paths).
#
# [Mar0819] Added folder-chooser prefills and starts as config-file options.
# These can often avoid the tedious Android Tk folder-chooser dialog.
#-------------------------------------------------------------------------------
A portable Python 3.X/2.X tkinter/Tkinter desktop GUI, for easily launching
mergeall.py's -report and -auto (but not selective/interactive) usage modes.
For screenshots of this GUI in action, see the examples/Screenshots folder.
Usage: This is a .pyw -- it runs with no console on Windows. Run this script
with no arguments, via icon clicks, command lines, or other. Drag it out to
a desktop shortcut for quick access. See folder docetc/launcher-configs for
a Windows desktop icon (attach it to a shortcut via right-click + Properties).
Uses widgets for choosing options and directory paths and viewing scrollable
mergeall stdout/stderr output, and supports saving and viewing mergeall's
output in log files. Uses threads to stay active while waiting for mergeall
output lines. Stays open to support multiple mergeall runs in one session --
report differences, run automatic updates, report on results, etc. The
underlying mergeall.py was not changed: this is just a GUI shell.
This is easier than launch-mergeall-Console.py (no directory typing, and
quicker options selection), which is easier than running raw mergeall.py
command lines directly and manually from a console or shortcuts, though
mergeall.py command lines support more options (e.g., interactive mode).
As of version 1.4, this also uses threading to always remain responsive.
As of version 2.0, this also supports the mergeall auto-backups option.
As of version 3.0, this also supports mergeall's "-skipcruft" and Mac OS X.
See below for major changes made here, as well as open issues (TBDs).
--------------------------------------------------------------------------------
VERSION 3.1 CHANGES:
These were internals and command-line changes, and had no GUI impact.
VERSION 3.0 CHANGES:
This major release's changes were largely driven by a Mac OS X port,
and new usage on Linux. Search for 'RunningOnMac' and '3.0' for Mac-ness.
Mergeall grew cruft file, symlink, and Windows long-path support in 3.0,
but most of these are not related to the GUI managed here.
App bar icon for Linux:
Linux now gets a nice mergeall app bar (launcher) icon. Mac icons
remain a TBD, and Windows has always had window icons for this program.
Initial desktop logs folder for both Linux and Mac:
The preset default for the logs folder now uses $HOME settings on these
Unix platforms (Windows uses the user's desktop as before).
Redesign run verification dialog for Linux and Mac:
The dialog popped up just before a run is launched was redesigned to
make it more readable on Mac and Linux. Its text was formerly fine
on Windows, but fairly bunched-up elsewhere. Also specialized the
warning text to be less dire if backups are enabled.
Top-level window hack for Mac Tk:
The Mac port required a top-level window hack to show buttons in Mac
active-window style initially, using the recommended Tk 8.5 install.
This is a complex story; see __main__ code in this file for details.
New toggle to suppress comparison lines, all platforms:
The Mac port also inspired a new GUI toggle to suppress folder comparison
message scrolling in the GUI (only). This is on initially for Mac because
the Mac Tk Text widget is VERY slow: ~30x slower than the mergeall process's
output. This toggle is off initially for Windows and Linux, because the GUI
largely keeps up on these platforms, and the messages indicate progress.
Still, the new toggle is available on these platforms too, because the GUI's
scrolling can add a few seconds to mergeall runtime in some tests run on
Windows, and may be a factor on slower machines; disable as desired.
Either way, full details, including all comparison messages, are always
available in saved log files popped up automatically at the end of a run.
Note that this new toggle differs from 2.4's "-quiet" option and GUI toggle
described below: "-quiet" disables output in mergeall itself before it ever
reaches the GUI, because backup messages are arguably distracting in logs
too; comparison messages are still useful in logs, if not the GUI.
Unlike all other widgets (except the new post-run popup toggle), this toggle
is also dynamic: setting it on/off while mergeall is running hides/shows
messages currently being generated. The 'skipping' message is displayed
every time this toggle is turned back on, though it does not appear until
the next mergeall message is received.
TBD: Mac text scrolls still seem painfully slow when there are many
difference-report lines. These could be also be suppressed, in addition
to folder comparison messages, by: line.startswith(('comparing', '[', ' ')).
This was ruled out, as it makes report-mode runs useless. Text speed must
be fixed by Tk's Mac developers (though one report on it went unreplied).
UPDATE: see docetc/miscnotes/mac-weirdly-slow-tk85-text-scroll.py for a
simple self-contained demo of the text scrolling slowness on ActiveState
Tk 8.5.18 (the Tk recommended by python.org), Python 3.5, and OS X 10.11.
It finishes scrolling in just 3-4 seconds on Windows, but 85 on Mac!
UPDATE: recent heroic efforts at speeding Mac Text widget scrolling came
up short. It's simply slow in all Mac Tks tested - 8.5.9 through 8.5.18.
The culprit seems to be update() calls required to redraw the widget after
new text is inserted. The code used to scroll uses normal techniques:
statustxt.insert(END, line) # add to Text widget
statustxt.see(END+'-2l') # reposition text
statustxt.update() # update display from within a loop
1) Alternative scroll techniques tried had no effect on speed:
statustxt.yview_scroll(2, 'units')
statustxt.yview_moveto(1.0)
statustxt.yview(END)
2) Using update_idletasks() speeds scrolls only slightly, and all the GUI's
controls are unresponsive while the text scrolls (an unacceptable effect):
statustxt.update_idletasks()
3) The after() timer loop's speed proved irrelevent, as the code mostly
stays in the 'batch' inner loop, and doesn't rescedule after() events:
statustxt.after(10, streamconsumer, linequeue, logfile, logpath)
4) The update() call can be avoided by processing just 1 line per after()
event (instead of the batch inner loop) with a very low delay count, but
this has NO impact on scroll speed - implicit updates are also too slow:
statustxt.after(1, streamconsumer, linequeue, logfile, logpath)
5) This leaves disabling scrolls (the new toggle), or updating only after
groups of lines are addded, which makes scrolling too jumpy and chaotic:
global upcnt
upcnt += 1
if upcnt % 50 == 0:
statustxt.update()
In the end, this is a Mac Tk bug, which hopefully is or will be addressed
in later Tks on the Mac. The new toggle is mergeall's best workaround.
New script to delete Mac ".*" cruft files:
The Mac platform has a habit of creating lots of metadata files and a few
folders, whose names all start with a ".", and which are sometimes treated
as hidden. These have meaning and purpose on a Mac, but are useless
noise on other platforms. For mergeall users on Windows and Linux whose
archive might become infected with these files by an association with a
Mac, a new script, "nuke-cruft-files.py," is provided to remove such files
as a pre-step or post-step to mergeall runs that copy archives off a Mac
See that script's extensive docstring for more details.
Disable widgets instead of erasing them, all platforms:
Also partly for the Mac, the GUI was redesigned to enable/disable widgets
as they fall in and out of relevance, instead of drawing/erasing them with
pack() and pack_forget(). The latter scheme causes a noticeable flash on a
Mac, but may have been too dramatic in general. The "GO" button didn't
cause flash, but enabling/disabling worked around an old text scroll issue.
New mode and toggle to skip system cruft files in both TO and FROM
Inspired by the numerous ".*" files added to archives on Mac OS X, both
mergeall and diffall grew a "-skipcruft" command line argument and mode,
which skips known cruft files of all platforms in both FROM and TO. The
GUI also grew a new toggle to suport this new mode and argument.
When enabled, the net effect is that cruft files do not register as
differences in report runs, and are not copied to, removed from, or replaced
in the TO tree in update runs. Such files thus stay on their generating
platform only - they won't be transferred to intermediate drives and other
computers, and won't be deleted from the generating platform by future
merges.
This option can be disabled, because users of a single platform may not
care about their cruft, and some crufts may be more useful than others.
Cruft files are defined in the mergeall_config.py file; users are
encouraged to adjust the set of matching files as needed for their use.
Sanitize Unicode characters in message-scroll text outside Tk's BMP range
Through 8.6, at least, Tk cannot display Unicode characers whose code
points are outside range U+0000..U+FFFF (BMP, UCS-2). Passing these to
Tk raises an uncayght exception which leaves the GUI in an unpredictable
state (usually half-drawn or hung). To work around this, replace all
such characters with the standard Unicode replacement character, which
renders as a generic indicator. This limit may be lifted in Tk 8.7.
Update: also an issue for pathnames in Browse folder dialogs, fixed here.
More descriptive GUI labels and popups
Toggle labels and ppups were given more explicit labels for clarity.
Allow editor popup to be disabled: config setting, new toggle
The automatic popup of a text editor to view a saved mergeall log file
can now be dsabled by a setting in mergeall_configs.py (on by default).
The popup may seem overkill to users who require a view-only display.
UPDATE: this is now switchable on/off by a new toggle in the GUI itself.
The configs-file setting is retained, but used only for an intial value
for the new toggle. The popup is normally unused, and GUI clicks are
easier than config edits. This new toggle is dynamic, like the message
scroll disabler; you can click it during a run to impact the popup.
Configurable message text display area
Users can now tailor the color, font, and initial size of the GUI's
scrolled display area for mergeall messages, in mergeall_configs.py.
The GUI's cosmetics were also polished in general along the way.
Mac changes to open dialogs (Browse for folders TO, FROM, Logs)
On Macs, use "message" ("title" is ignored), and add slide-down sheet
style (versus popup window) via "parent=root" if configs file setting.
Save dialogs seem to post titles correctly. Mac's standard menu is
customized here too, even though this program doesn't have one per se.
mergeall and cpall: copy unix symlinks, don't follow
Avoid redundant data copies. See mergeall.py and cpall.py for details.
Refocus on window after standard/common dialogs for Mac
Run a root.focus_force() after all standard dialogs used here to force
focus and active styling to be reset after dialog is closed. Else, Mac
users must click window after dialogs. This may be AS Tk 8.5 only; has
no effect on the initial style issue discussed earlier; and parent=root
doesn't suffice to reset focus in the standard dialogs used here.
TBD: should these dialogs also be made slide-down sheets on Mac like
the open/save diaogs via parent=root, instead of modal popup windows?
Some Mac purists may vote yea, but mergeall currently sides with variety,
especially given that the mergeall GUI is a single-window interface.
mergeall output forced to ASCII in PyInstaller Windows exes
A continuation of a long-running theme here, mergeall now forces stdout
text to be ASCII when running as a frozen executable on Windows ONLY.
See mergeall.py for details; this works around a likely PyInstaller bug.
--------------------------------------------------------------------------------
VERSION 2.4 CHANGES:
Added a toggle for the new "-quiet" script option that turns off per-file
"...backing up" log message printing in the underlying mergeall program.
These messages may make reading the log more work than it should be, and
don't add much information once backups are understood. The new "-quiet"
toggle in the GUI is active only when backups are enabled. Also tweaked
some GUI labels' text for clarity.
VERSION 2.2 and 2.3 CHANGES:
These versions were optimizations and patches, and had no impact on this GUI.
VERSION 2.1 CHANGES:
This version's changes were command-line only, and had no effect on this GUI.
--------------------------------------------------------------------------------
VERSION 2.0 CHANGES:
1) Added Backups toggle and corresponding mergeall -backup command-line arg.
The new backups frame retains state but is shown/hidden when -auto is
selected/deselected, as it's applicable to -auto only here (not -report).
In the console launcher, -backup is applicable to both -auto and
selective (=[not -report]) modes. Defaults to enabled in the GUI.
2) Moved log-file path chooser widgets down to log-file toggle button section.
The chooser retains state but is now shown/hidden when logging is toggled
on/off, as it's applicable when logging only.
3) Added "help" button that pops up the main usage doc in a web browser.
Packed in the run-mode frame's unused space.
4) The new "finished\n\n" message issued by mergeall solves the issue here
where the last output line was covered by repacking the GO button after resizes.
This works better, as it doesn't require the text display to scroll abruptly.
5) Use more descriptive text for the mode radio buttons (formerly was just
not "Report Only" and "Automatic Updates").
6) Error-check the FROM and TO paths here before trying to run mergeall, so
produces a GUI popup instead of mergeall text output message (log file path
was already being checked this way, because it required an open here).
7) Default to Desktop for log files on Windows (initial setting value only).
--------------------------------------------------------------------------------
VERSION 1.7.1 CHANGES:
None here (usage note, mergeall error message format change: see Revisions.html).
VERSION 1.7 CHANGES:
1) Minor bug fix for 2.X only -- add import of Tkinter's showerror when
using 2.X, else dialog never appears if bad log-file name.
2) Catch PermissionError (etc.) on log file open and report error in popup;
else fails silently on Windows, as ".pyw" has no console for exception text.
--------------------------------------------------------------------------------
VERSION 1.6 CHANGES: a new Python 2.X Unicode workaround, verify quits
Decoding an output line read from the spawned mergeall.py stream can fail
rarely in Python 2.X (only), if there are non-ASCII filename characters
in the line. This failure does not occur in 3.X. Patched to catch the
decode error and recover, with a notation at the front of the line in the
GUI. This change impacts text in the GUI display only; it does not impact
text in the log file, or the underlying mergeall process's run. Also added
a verify dialog for the window close button to avoid accidental exits.
[Discussion] The Unicode decode failure seems a fundamental 3.X/2.X
behavior difference. After initial research, the most likely culprits
appear to be:
a) The 2.X subprocess module mutates non-ASCII stream lines in transit.
b) The 2.X print statement generates either already-decoded text, or
wrongly-encoded bytes that do not respect the PYTHONIOENCODING setting.
In either case, this may reflect the fact that there is no notion of a
truly binary bytes stream in 2.X -- the content of subprocess output is
not as clearly defined as in 3.X's strict text/bytes dichotomy worldview.
Note that setting PYTHONIOENCODING in the shell does not fix this failure.
This environment variable is used for prints to the console, but this GUI
does only screen updates and log file writes itself, and exports this
setting automatically to the mergeall subprocess for its prints. Testing
verifies that os.environ settings are inherited by 2.X subprocesses too,
even when spawned by the subprocess module.
Whatever the exact cause, this is a fundamental 2.X/3.X difference and
another 2.X/3.X semantic incompatibility that goes well beyond syntax.
In this case, adding an exception handler around the offending decode
fixed the failure for all cases tested in 2.X, so this issue is closed.
Given the other 2.X/3.X incompatibilities found and addressed in this
project, though, it seems that writing dual-version 2.X+3.X code may be
a bit of a pipedream when it comes to realistic, practical programs.
If in doubt, try running mergeall on Python 3.X instead of 2.X.
Also note that this patch applies only to the GUI launcher: PYTHONIOENCODING
must still generally be set manually in your system shell when running either
the console launcher or script mergeall.py directly from a command line, if
either may ever process and thus print non-ASCII filenames. Else, such
filenames may cause both scripts to abort, especially in Python 3.X. The
GUI launcher does not require this setting, as it automatically sets and
propagates this variable to its mergeall.py subprocess, and never prints.
[3.0] UPDATE: for its PyIstaller frozen executable ONLY, mergeall must force
stdout text to ASCII, which isn't ideal, but is requied in this context to
avoid exceptions, and sidesteps this entire display-only issue.
--------------------------------------------------------------------------------
VERSION 1.5 CHANGES: Linux port and usage
Pass shell=False on Linux/Unix (only) to subprocess.Popen, else starts a
"python" interactive shell even though a full command sequence is passed.
Linux users: see also release 1.5 notes in docs/Revisions.html for "#!"
pointers, and a possible NTFS timestamp skew issue for Windows/Linux
cross-platform syncs.
--------------------------------------------------------------------------------
VERSION 1.4 CHANGES: threads, streams, log files
1) Threading
Add threading for the mergeall subprocess stream reader, a former TBD.
This structure is more complex -- it trades a simple loop for two functions
and multiple levels of loops -- but prevents the GUI from becoming blocked
and unresponsive while waiting for a next line from the subprocess.
This was normally not an issue: the GUI updates after each new line, and is
used just for viewing after starting a mergeall run (and it's "GO" button is
erased during this process to avoid overlapping runs). However, blocked
states are not natural in GUIs, and they can become apparent here if mergeall
is busy copying large trees. See book (PP4E) for more details and examples
of GUI threading.
2) Stream Unicode decoding, take 2
Force UTF8 encoding for prints in mergeall subprocess via PYTHONIOENCODING,
and use binary mode Popen stream reads + manual post-read UTF8 decoding here.
Version 1.2 formerly made the subprocess's streams encoding match Popen's
expectation (cp1252 on Windows) by using locale.getpreferredencoding(False);
that works for reading the stream, but not for prints within mergeall itself.
3) 2.X log-file compatibility
Use binary mode for the log files, writing the now-binary stream data.
Former versions allowed for non-ASCII filenames in the log file text by
using text mode and writing characters as UTF8, but the 2.X codecs.open()
doesn't expand \n the same as 3.X's text-mode open() (see also next item).
4) 2.X unbuffered streams compatibility
Temporarily dropped '-u' in mergeall spawn command-line, as it makes eolns
(linebreak character sequences) \n in 2.X, but \r\n in 3.X. This causes
issues in log-file writes: without a \r\n on Windows, files are single lines.
LATER UPDATE: the Python '-u' unbuffered flag has been reinstated. Without
it, mergeall output may not appear for 10 or more seconds on some machines
and slower devices due to internal buffering. Because this flag also makes
line-breaks differ between Python 2.X and 3.X, though, also need to use
special-case log-file writes to map all linebreaks to the platform's version.
This is a complex 3.X/2.X compatibility issue, involving -u, Popen, and opens.
Neither Popen text mode streams nor the 2.X codecs.open() will help. The
former can't be used because its internal encoding policy (per locale) is
not broad enough to handle arbitrary Unicode filenames, and the latter always
opens in binary mode, and so does no translations of linebreaks in the decoded
text (3.X's text-mode open() does expand linebreaks by default).
Could write binary lines in 2.X text-mode open() to expand \n on Windows,
but that won't work in 3.X -- its text-mode open() expects Unicode strings,
and always does encoding in addition to linebreaks. Writing manually
encoded text via open() in 3.X and codecs.open() in 2.X also won't work:
2.X requires manual \r\n, which 3.X will by default expand to \r\r\n.
3.X open() supports a 'newline' argument to turn off \n expansion, but
this can't be used either, as it's not available in 2.X's codecs.open().
As in, *punt*: write binary data with manual \n mapping for log files.
--------------------------------------------------------------------------------
TBD 1: No selective/interactive mode in GUI
mergeall's interactive mode is unavailable here as is, due to an outstanding
TBD regarding handling and interleaving of stdout for input() prompts in spawned
Python 3.X processes. In practice, though, the -report and -auto modes have been
the only modes regularly used. See launch-mergeall-Console.py for details on the
issue (the console launcher supports interactive mode, but without a log file).
TBD 2: 2.X compatibility for Unicode filenames?
This system works well on 3.X (it's main usage platform) and is largely 2.X
compatible, but the launchers may still have issues in some stream decoding
for non-ASCII filenames. No such encoding exceptions occur on 2.X for the
raw mergall.py script, though some non-ASCII os.listdir results seem a bit
suspect in 2.X too. More complete 2.X testing remains suggested exercise.
--> Update: see also the 1.6 decoding change note above. <--
TBD 3: Decoupled versus single-process models?
The launchers currently use a standard decoupled model that spawns mergeall
and reads and decodes its streams. There may be advantages to using a
single-process model that instead imports and calls mergeall's functions.
See docs\Lessons-Learned.html for more discussion on this alternative.
================================================================================
"""
#from __future__ import print_function # 2.X compatibility: not needed here
APPNAME = 'mergeall'
VERSION = 3.1
# [3.0] for frozen app/exes, fix module+resource visibility (sys.path)
import fixfrozenpaths
import sys
if sys.version[0] >= '3': # Py 3.X, but allow for Py 4.X too [3.0]
import _thread, queue
from tkinter import *
from tkinter.messagebox import askokcancel, showinfo, showerror
from tkinter.filedialog import Directory # saves last dir
from tkinter.scrolledtext import ScrolledText
else:
import thread as _thread, Queue as queue # Py 2.X compatibility
from Tkinter import *
from tkMessageBox import askokcancel, showinfo, showerror # [1.7]
from tkFileDialog import Directory
from ScrolledText import ScrolledText
#import codecs
#open = codecs.open # [1.4] log binary mode from stream, not text files
# this script isn't too platform-specific, but avoid repeating this
RunningOnMac = sys.platform.startswith('darwin')
RunningOnWindows = sys.platform.startswith('win')
RunningOnLinux = sys.platform.startswith('linux')
import webbrowser, subprocess, time, os
# [3.0] data+scripts not in os.getcwd() if run from a cmdline elsewhere,
# and __file__ may not work if running as a frozen PyInstaller executable;
# use __file__ of this file for Mac apps, not module: it's in a zipfile;
MYDIR = fixfrozenpaths.fetchMyInstallDir(__file__) # absolute
# [3.0] new doc, in this script's folder - but not necessarily '.' (cwd)
HELPFILE = os.path.join(MYDIR, 'UserGuide.html')
# [3.0] Mac OS X is pickier about file URLs
if RunningOnMac:
HELPFILE = 'file:' + HELPFILE
# ANDROID [Apr1219] - open latest online version of user guide (all platforms should)
HELPURL = 'https://learning-python.com/mergeall-products/unzipped/UserGuide.html'
# [3.0] part of PP4E's guimaker module, copied here to avoid dependency
from guimaker_pp4e import fixAppleMenuBar
# [3.0] user configs: scrolled messages text area, log-file popup;
# for GUI settings, None = use Tk defaults: 24 lines high, 80 chars wide;
try:
from mergeall_configs import (
LOGEDITORPOPUP,
DEFAULTLOGDIR,
MACSLIDEDOWN,
TEXTAREAHEIGHT, TEXTAREAWIDTH, TEXTAREAFONT, TEXTAREACOLOR,
# ANDROID [Mar0819]
DEFAULTFROMDIR, DEFAULTTODIR, # entry prefills: see usage ahead
BROWSELOGDIR, BROWSEFROMDIR, BROWSETODIR, # chooser-dialog starts: ditto
# ANDROID [Mar2819]
LABELFONT, # font if labels too wide to fit
HEADERFONT) # custom font for section headers
except Exception as why:
# if any fail, all default (brutal, but simple)
LOGEDITORPOPUP = True # default: initial value for log-file popup toggle
DEFAULTLOGDIR = None # default: Desktop folder?, per running platform
MACSLIDEDOWN = False # default: popup, not sheet, for folder dialogs
TEXTAREAHEIGHT = 20 # initial number lines in message scroll widget
TEXTAREAWIDTH = None # initial number characters per line, wrapped
TEXTAREAFONT = None # scrolled messages font, None=Tk default font
TEXTAREACOLOR = None # scrolled messages color(s), None=Tk default font
# ANDROID [Mar0819]
DEFAULTFROMDIR = None # default: always Browse in GUI (entry prefill)
DEFAULTTODIR = None # ditto
BROWSELOGDIR = None # default: use tkinter start default (chooser dialog)
BROWSEFROMDIR = None # ditto
BROWSETODIR = None # ditto
# ANDROID [Mar2819]
LABELFONT = None # default to system default: best on most devices
HEADERFONT = None # default to small preset in code ahead
print('Error in config file: %s' % why) # to console, if any
def fixTkBMP(text):
"""
[3.0] (copied from PyMailGUI) Tk <= 8.6 cannot display Unicode characters
outside the U+0000..U+FFFF BMP (UCS-2) code-point range, and generates
uncaught exceptions when tried (emojis kill programs!). To address this,
call this function to sanitize all text passed to the GUI for display.
It replaces any non-BMP characters with the standard Unicode replacement
character U+FFFD, which Tk displays as a highlighted question mark diamond.
This workaround is coded to assume that Tk 8.7 will lift the BMP restriction,
per a dev rumor. It also assumes TkVersion has been imported from tkinter.
Use here: filenames in mergeall messages scrolled text (rare, but true).
"""
if TkVersion <= 8.6:
text = ''.join((ch if ord(ch) <= 0xFFFF else '\uFFFD') for ch in text)
return text
def isNonBMP(text):
"""
[3.0] Return true if any char (codepoint) in text is outside Tk's BMP range.
Used by folder dialogs to force initialfile=None when True for prior choice.
"""
if TkVersion <= 8.6:
return any(ord(ch) > 0xFFFF for ch in text)
else:
return False # and assume Tk 8.7 will make this better...
def refocusWindow():
"""
[3.0] Call after (most) standard dialogs to reset focus on the main
window, else focus and active style are lost when dialog is closed.
This may be a bug in ActiveState Tk 8.5 (TBD), but the fix is simple.
"""
root.focus_force() # else Mac requires a user click after dialogs
####################################################################################
# GUI BUILDER
####################################################################################
# font for headers in controls sections;
# could be user-configurable too, but seems arguably-better hardwired;
# [2.4] not just 'bold': if used as family name, Tk falls back on arial!
# [3.0] smaller font on Linux, else looks almost cartoonish;
HDRFONT = 'arial 14 bold' # ('family', size, 'style? style?...')
if RunningOnLinux:
HDRFONT = 'arial 12 bold' # smaller is better on Linux (not Mac, Windows)
# ANDROID - go smaller (arial==helvetica, which supports bold in font str or tuple);
# [Mar2819] also allow user to give header font in config file, along with label font;
#
HDRFONT = HEADERFONT or 'arial 6 bold' # use small default if None
def makewidgets(root):
"""
build the gui window, setup Browse/GO/other event handlers
"""
# used on GO
global dirents # folders
global modevar, logvar, bkpvar # settings
global quietvar, cmpmsgsvar, cruftvar, logpopupvar # more settings
global gobutton, statustxt # widgets
#---------------------------------------------------------------------------
# Event handlers (less ongobutton: ahead)
#---------------------------------------------------------------------------
# some use names in the enclosing func (which are actually globals)
# one open dialog for entire run, saved in scope (closure)
opendirdlg = Directory()
def onbrowse(field, label):
title = 'Choose mergeall %s folder' % label
if RunningOnMac: # [3.0] specialize on Mac
if MACSLIDEDOWN:
# Mac: message (title ignored), slidedown window, custom text
title = 'Choose mergeall %s folder' % label
dlgkargs = dict(message=title, parent=root)
else:
# Mac: message (title ignored), popup window, standard text
title = 'mergeall: Choose %s Folder' % label
dlgkargs = dict(message=title)
else:
# Windows+Linux: the usual modal popup, standard text
title = 'mergeall: Choose %s folder' % label
dlgkargs = dict(title=title)
# check prior pathname choice for emojis: kills dialog
prior = opendirdlg.options.get('initialdir', '')
if isNonBMP(prior):
dlgkargs.update(dict(initialdir=None)) # for this call only
# ANDROID [Mar0819] - use config starts if set because the Android tkinter
# chooser GUI is tedious+slow (like its Linux cousin, but worse on phones);
# Android uses different settings for entry prefills and dialogs starts;
#
starts = dict(FROM=BROWSEFROMDIR, TO=BROWSETODIR, Logs=BROWSELOGDIR)
if starts[label] and os.path.isdir(starts[label]):
dlgkargs['initialdir'] = starts[label]
else:
pass # use tkinter's default
# or: dlgkargs['initialdir'] ='/' # dlgkargs.update() is overkill here
# dlgkargs['initialdir'] = None # this leaves None and kills isNonBMP
chosendir = opendirdlg.show(**dlgkargs)
if chosendir:
field.delete('0', END)
field.insert(END, fixTkBMP(chosendir))
refocusWindow() # [3.0] else requires click on Mac
def onquit():
answer = askokcancel('%s: Verify Exit' % APPNAME, 'Really quit mergeall now?')
if answer:
win.quit() # win in enclosing scope; or sys.exit()
else:
refocusWindow() # [3.0] else requires click on Mac
def onmodetoggle():
if modevar.get().startswith('REPORT'): # hide/show backups frame [2.0]
# [3.0] bkpfrm.pack_forget() # -backup applies to -auto updates only
bkpbtn.config(state=DISABLED) # enabled/disable widgets [3.0]
quietbtn.config(state=DISABLED)
cruftbtn.config(text='Skip cruft items in FROM and TO?') # ANDROID - shorter
#'do not report as differences?') # ANDROID
else:
# [3.0] bkpfrm.pack(expand=YES, fill=X)
bkpbtn.config(state=NORMAL)
quietbtn.config(state=NORMAL)
cruftbtn.config(text='Skip cruft items in FROM and TO?') # ANDROID - shorter
#'do not copy, replace, or delete?') # ANDROID
def onlogtoggle():
if logvar.get(): # show/hide log-file path chooser [2.0]
# [3.0] logdirfrm.pack(expand=YES, fill=X) # chooser applies only if logging
logbtn.config(state=NORMAL)
logent.config(state=NORMAL)
logpopupbtn.config(state=NORMAL) # [3.0] ditto for log-popup toggle
#logpopupbtn.config(state=DISABLED) # ANDROID - webbrowser failed initially
else:
# [3.0] logdirfrm.pack_forget()
logbtn.config(state=DISABLED)
logent.config(state=DISABLED)
logpopupbtn.config(state=DISABLED)
def onbkptoggle():
if bkpvar.get(): # show/hide -quiet toggle button [2.4]
# [3.0] quietbtn.pack(anchor=NW) # -quiet applies only if doing backups,
quietbtn.config(state=NORMAL) # and whether saving to log file or not
else:
# [3.0] quietbtn.pack_forget()
# quietvar.set(False) # keep prior setting, disabled=moot [3.0]
quietbtn.config(state=DISABLED)
def onhelp():
#
# ANDROID [Apr1219]: webbrowser fails on Android (for reasons TBD),
# so spawn a shell command using the $BROWSER preset in Pydroid 3:
# "am start --user 0 -a android.intent.action.VIEW -d %s"; Android
# uses online version to pick up latest changes (others should too);
#
# ANDROID [Apr1919]: webbrowser _does_ work, but requires local file
# URLs to start with "file://" and does not open a web browser for
# local HTML files (they open in text editors); use the online URL
# to ensure a web browser, and either os.system or webbrowser.open;
#
# ANDROID [Apr2119]: Pydroid 3 3.0 broke webbrowser and changed
# $BROWSER - use os.system() with a hardcoded command-line string;
#
brw = 'am start --user 0 -a android.intent.action.VIEW -d %s'
cmd = brw % HELPURL
os.system(cmd) # was os.environ['BROWSER'], webbrowser.open(HELPURL)
# other platforms code...
"""
webbrowser.open(HELPFILE) # popup local file in web browser
"""
#---------------------------------------------------------------------------
# Build the GUI (link to handlers)
#---------------------------------------------------------------------------
#
# MAIN WINDOW
#
win = root # [3.0] allow for TopLevel() or Tk()
win.title('mergeall %.1f' % VERSION) # set main window title, [1.6] version
win.protocol('WM_DELETE_WINDOW', onquit) # [1.6] catch/verify window close
# replace red (no, blue...) tk icon?
iconfolder = os.path.join(MYDIR, 'icons')
try:
if RunningOnWindows:
# try Windows window icon
icnpath = os.path.join(iconfolder, 'mergeall.ico')
win.iconbitmap(icnpath)
elif RunningOnLinux:
# try Linux app-bar icon [3.0]
icnpath = os.path.join(iconfolder, 'mergeall.gif')
imgobj = PhotoImage(file=icnpath)
win.iconphoto(True, imgobj)
elif RunningOnMac or True:
# Mac OS X: neither of the above works (yet?) [3.0]
# Macs require apps for most icon contexts
raise NotImplementedError
except Exception as why:
# punt: bad file/call or platform (Mac OS X TBD)
pass
# [3.0] on Mac, customize app-wide automatic top-of-display menu
fixAppleMenuBar(root, 'mergeall',
helpaction=onhelp, aboutaction=None, quitaction=onquit)
ctrlfrm = Frame(win)
ctrlfrm.pack(expand=NO, fill=BOTH, side=TOP)
#
# MAIN TO/FROM DIRECTORY CHOOSERS
#
dirsfrm = Frame(ctrlfrm, relief=GROOVE, border=3)
dirsfrm.pack(fill=X)
Label(dirsfrm, text='Main Folders', font=HDRFONT).pack() # [2.4] font
rowsfrm = Frame(dirsfrm)
rowsfrm.pack(expand=YES, fill=X)
rowsfrm.columnconfigure(1, weight=1)
dirents = {}
for (row, key) in enumerate(('FROM', 'TO')):
rowsfrm.rowconfigure(row, weight=1)
Label(rowsfrm,
font=LABELFONT, # ANDROID - configurable [Mar2819]
text=key + ' folder:').grid(row=row, column=0, sticky=E)
dirent = Entry(rowsfrm)
dirent.insert(END, 'enter or browse...')
dirent.grid(row=row, column=1, sticky=EW)
handler = lambda dirent=dirent, key=key: onbrowse(dirent, key) # current!
Button(rowsfrm, text='Browse...',
command=handler).grid(row=row, column=2)
dirents[key] = dirent
# ANDROID [Mar0819] - prefill to avoid tedious Android/phone chooser dialog
for (key, prefill) in [('FROM', DEFAULTFROMDIR), ('TO', DEFAULTTODIR)]:
if prefill and os.path.isdir(prefill):
dirents[key].delete('0', END)
dirents[key].insert(END, prefill)
#
# RUN MODE RADIO BUTTONS: -report or -auto
#
radiofrm = Frame(ctrlfrm, relief=GROOVE, border=3)
radiofrm.pack(fill=X)
Label(radiofrm, text='Run Mode', font=HDRFONT).pack()
# help button: use empty space in run mode frame [2.0]
# ANDROID [Apr1219]: use default color to avoid loss, was [relief=FLAT, bg='white')]
#
helpbtn = Button(radiofrm, text='Help', command=onhelp)
helpbtn.pack(side=RIGHT, anchor=NE)
#helpbtn.config(state=DISABLED) # ANDROID - webbrowser failed initially
modevar = StringVar()
modes = ['REPORT: show differences only', # ANDROID - shorter
'UPDATE: make TO the same as FROM'] # ANDROID - shorter
for mode in modes:
Radiobutton(radiofrm,
text=mode,
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=modevar,
value=mode,
command=onmodetoggle).pack(side=TOP, anchor=NW)
modevar.set(modes[0])
# [3.0] skip cruft files in FROM and TO - don't copy/remove/replace or report
cruftvar = BooleanVar()
cruftvar.set(True)
cruftbtn = Checkbutton(radiofrm,
text='Skip cruft items in FROM and TO?', # ANDROID - shorter
#'do not report as differences?', # ANDROID
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=cruftvar)
cruftbtn.pack(side=LEFT)
#
# MESSAGE TOGGLES + LOG DIRECTORY CHOOSER
#
msgfrm = Frame(ctrlfrm, relief=GROOVE, border=3)
msgfrm.pack(fill=X)
Label(msgfrm, text='Messages', font=HDRFONT).pack()
logfrm = Frame(msgfrm)
logfrm.pack(side=TOP, fill=X)
logvar = BooleanVar() # Intvar works too
logvar.set(True) # else default=off (eibti)
Checkbutton(logfrm,
text='Save logfile to folder? ', # ANDROID - shorter
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=logvar,
command=onlogtoggle).pack(side=LEFT)
# log-file dir chooser by toggle, unhide/hide when toggled on/off [2.0]
key = 'Logs'
logdirfrm = Frame(logfrm)
logdirfrm.pack(expand=YES, fill=X)
handler = lambda: onbrowse(logent, key) # last, not current (!) [see above]
logbtn = Button(logdirfrm, text='Browse...', command=handler)
logbtn.pack(side=RIGHT)
logent = Entry(logdirfrm)
logent.insert(END, 'enter or browse...') # ANDROID - shorter
logent.pack(side=LEFT, expand=YES, fill=X)
dirents[key] = logent
# [3.0] logdirfrm.pack_forget() # till logs toggled on (now enabled/disabled)
logbtn.config(state=NORMAL) # till logs toggled on (initially are)
logent.config(state=NORMAL)
# fill initial/default logs folder value
if DEFAULTLOGDIR and os.path.exists(DEFAULTLOGDIR):
# [3.0] allow user-configs file to give initial default
defaultlogs = DEFAULTLOGDIR
else:
try:
# [2.0] try user's Desktop on Windows (has HOMEPATH but no HOME)
defaultlogs = r'C:\Users\%s\Desktop' % os.environ['username']
assert os.path.exists(defaultlogs)
except:
try:
# [3.0] try same on Linux and Mac OS X (TBD: or Documents?)
defaultlogs = os.path.join(os.environ['HOME'], 'Desktop')
assert os.path.exists(defaultlogs)
except:
# ANDROID - try internal-storage docs folder (on some phones);
# [Mar0819] don't use /sdcard/Documents as a prefill (made by
# MS Office apps); create the preset if it doesn't exist; but
# still use DEFAULTLOGDIR as the prefill instead, if it's set;
#
try:
defaultlogs = '/sdcard/Admin-Mergeall' # was '/sdcard/Documents'
if not os.path.exists(defaultlogs): # make now if needed
os.mkdir(defaultlogs)
assert os.path.isdir(defaultlogs) # exists + is a folder
except:
defaultlogs = None
if defaultlogs:
# may be unset or fail on some Unix and Windows => empty, Browse
logent.delete('0', END)
logent.insert(END, defaultlogs)
# [3.0] allow comparison messages to be suppressed (mostly for Mac speed);
# this toggle (only) is dynamic: can change effect while output scrolling;
cmpmsgsvar = BooleanVar()
if RunningOnMac:
cmpmsgsvar.set(True) # initial=suppress on Mac OS X
else:
cmpmsgsvar.set(False) # off on Windws/Linux: fast GUI
Checkbutton(msgfrm,
text='Hide comparison messages?', # ANDROID - shorter [Mar2819]
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=cmpmsgsvar).pack(side=LEFT, anchor=NW)
# [3.0] allow log-file popup editor to be suppressed in the GUI per run;
# the configs-file entry added previously now gives an initial value only;
logpopupvar = BooleanVar()
logpopupvar.set(LOGEDITORPOPUP)
logpopupbtn = Checkbutton(msgfrm,
text='Popup logfile?', # ANDROID - shorter [Mar2819]
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=logpopupvar)
logpopupbtn.pack(side=RIGHT, anchor=NE)
#logpopupvar.set(False) # ANDROID - webbrowser failed initially
#logpopupbtn.config(state=DISABLED) # ANDROID - webbrowser failed initially
#
# BACKUP TOGGLES
#
# enable/disable when '-auto' run mode selected/deselected [2.0] [3.0]
bkpfrm = Frame(ctrlfrm, relief=GROOVE, border=3)
bkpfrm.pack(fill=X)
Label(bkpfrm, text='Backups', font=HDRFONT).pack()
bkpvar = BooleanVar()
bkpbtn = Checkbutton(bkpfrm,
text='Save TO items replaced or deleted, note adds?', # ANDROID - shorter [Mar2819]
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=bkpvar,
command=onbkptoggle)
bkpbtn.pack(anchor=NW)
bkpvar.set(True) # initial=on: do backups
# [3.0] bkpfrm.pack_forget() # till -auto selected
bkpbtn.config(state=DISABLED) # till -auto selected
# [2.4] support -quiet mode: omit "...backing up" per-file log messages
quietvar = BooleanVar()
quietbtn = Checkbutton(bkpfrm,
text='Disable per-item backup messages?', # ANDROID - shorter [Mar2819]
font=LABELFONT, # ANDROID - configurable [Mar2819]
variable=quietvar)
quietbtn.pack(anchor=NW)
quietvar.set(True) # init=suppress (though good for errors+feedback)
# [3.0] quietbtn.pack_forget() # only if backups selected, but will be initially
quietbtn.config(state=DISABLED) # till -auto AND -backups selected
#
# MESSAGES SCROLLED TEXT + 'GO' BUTTON
#
# 'go' always hidden during run to prevent overlapping run launches
statustxt = ScrolledText(win)
statustxt.config(state=DISABLED) # ANDROID [Mar2319]: readonly text to avoid keyboard
# [3.0] user configs: size, font, color (no border - color sets off better)
try:
if TEXTAREAHEIGHT: statustxt.config(height=TEXTAREAHEIGHT)
if TEXTAREAWIDTH: statustxt.config(width=TEXTAREAWIDTH)
if TEXTAREAFONT: statustxt.config(font=TEXTAREAFONT)
if TEXTAREACOLOR:
if isinstance(TEXTAREACOLOR, str):
statustxt.config(bg=TEXTAREACOLOR)
elif isinstance(TEXTAREACOLOR, tuple):
statustxt.config(bg=TEXTAREACOLOR[0])
statustxt.config(fg=TEXTAREACOLOR[1])
else:
print('Bad color value in config file')
except Exception as why:
print('Bad config setting: %s' % why)
gobutton = Button(win, text='GO: run mergeall', font=HDRFONT,
command=ongobutton)
gobutton.pack(side=BOTTOM)
statustxt.pack(side=TOP, expand=YES, fill=BOTH) # pack last=clip first
####################################################################################
# ON "GO": SPAWN MERGEALL PROCESS
####################################################################################
# [1.4] how spawned mergeall subprocess's text is written and decoded here
STREAM_ENCODE = 'utf8'
EOF_SENTINEL = [] # stream lines read will never be a list
linequeue = queue.Queue() # infinite-size shared queue of objects
def ongobutton():
"""
on GO button press: fetch gui values, confirm run, launch mergeall
"""
# set in makewidgets
global dirents # folders
global modevar, logvar, bkpvar, quietvar, cruftvar # settings
global gobutton, statustxt # widgets
# [3.0] for comparison message suppresssion
global firstcompareline
firstcompareline = True # reset before each run
# get inputs from GUI
fromdir = dirents['FROM'].get() # directory fields
todir = dirents['TO'].get()
logdir = dirents['Logs'].get()
mode = modevar.get() # runmode radiobtn
dolog = logvar.get() # logfile checkbtn
dobkp = bkpvar.get() # backups checkbtn
quiet = quietvar.get() # quiet mode checkbtn [2.4]
docruft = cruftvar.get() # skip cruft files [3.0]
# config run
modearg = '-report' if mode.startswith('REPORT') else '-auto'
if dobkp and modearg != '-report':
modearg += ' -backup' # [2.0] backup replacements/removals
if quiet:
modearg += ' -quiet' # [2.4] no per-file backup log messages
if docruft:
modearg += ' -skipcruft' # [3.0] ignore cruft files in FROM and TO
if not dolog:
logpath = logfile = None
else:
datestamp = time.strftime('date%y%m%d-time%H%M%S')
logpath = logdir + os.sep + 'mergeall-%s.txt' % (datestamp)
# confirm run
if modearg.startswith('-report'):
runtype = 'REPORT-ONLY RUN'
warning = 'This run will not change your data.'
else:
runtype = 'AUTO-UPDATE RUN'
warningbase = (
# ANDROID [Mar2319] - manually line break for fit (but Tk still truncates paths)
#
# '*WARNING*: by design, this may change your TO folder tree in-place, '
# 'by adding, replacing, and deleting files and folders as needed '
# 'to make TO the same as FROM.')
#
'*CAUTION*: by design, this run may change\n'
'your TO folder tree in-place, by adding,\n'
'replacing, and deleting files and folders\n'
'as needed to make TO the same as FROM.')
warningmore = (
# ANDROID [Mar2319] - manually line break for fit (but Tk still truncates paths)
#
# ' Because backups are disabled, any such changes '
# 'will be permanent and irrevocable.')
#
'\nBecause backups are disabled, any such\n'
'changes cannot be undone.')
warning = warningbase + ('' if dobkp else warningmore)
confirm = askokcancel('%s: Confirm Run' % APPNAME,
'About to run:\n'
'mergeall.py %s\n\n'
'FROM:\n%s\n\n'
'TO: \n%s\n\n'
'Logging output to:\n%s\n\n'
'%s\n\n'
'Start this %s?' %
(modearg, fromdir, todir, logpath, warning, runtype))
notruntitle = '%s: Not Run' % APPNAME
if not confirm:
showinfo(notruntitle, 'The mergeall run was cancelled.')
elif not os.path.exists(fromdir):
showerror(notruntitle, 'Please select a valid mergeall FROM folder.') # [2.0] popup
elif not os.path.exists(todir):
showerror(notruntitle, 'Please select a valid mergeall TO folder.') # [2.0] popup
elif dolog and not os.path.exists(logdir):
showerror(notruntitle, 'Please select a valid mergeall log file folder.') # or sooner?
else:
# [1.4] log uses binary mode for now-binary data from stream
if dolog:
try:
logfile = open(logpath, 'wb')
except:
# [1.7] catch PermissionError and show popup (else silent for .pyw)
showerror(notruntitle, 'Please select a writeable log file folder.')
return
# proceed with mergeall
# [3.0] gobutton.pack_forget() # hide/erase button
gobutton.config(state=DISABLED) # keep but disable
statustxt.config(state=NORMAL) # ANDROID [Mar2319]: enable text for changes during run
statustxt.delete('1.0', END) # clear last run text
# [1.4] force UTF8 prints in mergeall, use binary streams + manual decode here;
# this setting is inherited by the spawned mergeall subprocess for its prints;
os.environ['PYTHONIOENCODING'] = STREAM_ENCODE
# config mergeall command (sequences: auto-quoted by subprocess)
extras = {}
if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
# pyinstaller exe [3.0]
# run frozen executable directly, not script through python
freezename = 'mergeall.exe' if RunningOnWindows else 'mergeall'
mergeallpath = os.path.join(MYDIR, freezename)
os.environ['PYTHONUNBUFFERED'] = 'True' # -u equiv (iff env?)
cmdseq = [mergeallpath, # frozen executable
fromdir, # '/' ok on Windows
todir] + modearg.split() # 'a b?' -> ['a', 'b'?]
if RunningOnWindows:
# else spawn hangs unless launcher uses --console (with popup!)
# startupinfo, env=os.environ, and creationflags are irrelevant
extras.update(stdin=subprocess.DEVNULL)
else:
# py2app Mac app or source (original code)
# relative script path works whether run here or via desktop shortcut
# [3.0] but not in os.getcwd() if run from a cmdline elsewhere
# ANDROID - hardcode sys.executable, else empty in Pydroid 3: kills Popen
#
# ANDROID [Apr1919]: Pydroid 3's 3.0 release moved its Python from the
# first of the following paths to the second, breaking this workaround:
# /data/user/0/ru.iiec.pydroid3/files/arm-linux-androideabi/bin/python
# /data/user/0/ru.iiec.pydroid3/files/aarch64-linux-android/bin/python
# to allow for both paths--and be platform agnostic in general--read the
# result of a 'which python' shell command instead of using literal strs;
#
sys.executable = os.popen('which python').read().rstrip() # path to Python exe
scriptname = 'mergeall.py'
mergeallpath = os.path.join(MYDIR, scriptname)
cmdseq = [sys.executable, '-u', # [1.4] need -u unbufferred
mergeallpath, # script file (app or source)
fromdir, # '/' ok on Windows
todir] + modearg.split() # 'a b?' -> ['a', 'b'?]
# [1.5] shell should be True on Windows so that it uses filename associations,
# but False on Linux so that it doesn't just start a "python" interactive shell
doshell = RunningOnWindows
# spawn mergeall
# use subprocess: os.popen/spawnv not enough, popen2 is 2.X only;
subproc = subprocess.Popen(
cmdseq, # a string cmd may fail on Unix
shell=doshell, # [1.5] see note above, platform specific
universal_newlines=False, # [1.4] binary mode, manual decode/eoln
stdout=subprocess.PIPE, # capture sub's stdout here
stderr=subprocess.STDOUT, # route sub's stderr to its stdout
**extras)
# read and process mergeall's output: reader thread + timer-based poller
_thread.start_new_thread(streamreader, (subproc.stdout, linequeue))
streamconsumer(linequeue, logfile, logpath)
# returns here immediately: a thread and timer-event loop are now running
# [3.0] for all cases, else requires click on Mac
refocusWindow()
####################################################################################
# MERGEALL PROCESS HANDLER: THREAD + POLLER
####################################################################################
def streamreader(stream, linequeue):
"""
[1.4] In a parallel thread - read the mergeall subprocess's stdout/stderr
stream, and post its lines to a queue for the GUI to read and display
on timer event callbacks; this way, the GUI isn't blocked/paused during
long-running copies or other actions in the spawned mergeall script;
"""
for line in stream: # read stdout+stderr lines: may block this thread
linequeue.put(line) # place on queue to be picked up by GUI thread
linequeue.put(EOF_SENTINEL) # write sentinel at eof and exit: subprocess ended
def streamconsumer(linequeue, logfile, logpath):
"""
[1.4] In the main GUI thread - run a timer-based loop to poll for, read,
and display and log stream lines from the shared thread queue until the
reader thread sends the end-signal sentinel value on the queue. The main
GUI thread running this code thus remains active between mergeall output
lines. A nested loop is also used here to process lines in batches so the
GUI's response to lines is quick, but it calls update() to remain active.
At this point we have 2 threads and a process (connected by queue and stream)
and 3 or 4 loops going at once -- the main GUI thread runs a timer event loop
to poll the queue, and runs the nested line-batch loop here; the spawned
stream-reader thread runs a blockable reading loop and posts lines to the
queue; and the spawned mergeall process runs its own file-processing loops
to create output lines eventually displayed in the GUI here.
"""
global statustxt, gobutton # widgets
global firstcompareline # state
global cmpmsgsvar, logpopupvar # settings
def trydecode(binline, stream_encode):
"""
[1.6] line decode can fail in Python 2.X due to a TBD library
incompatibility issue; see 1.6 note near top of this script;
if uncaught, GUI is dead but unclosed, and the error message does
not appear in GUI -- because the error occurs here in GUI instead
of subproc, its text goes to console here (if any), not to subproc
stream queue, and the subproc is apparently terminated in 2.X;
also note that "line" text here is used for the GUI display only:
it doesn't show up in the binary log file, and this has no impact
on the underlying mergeall process, which proceeds unaffected;
[3.0] changed the error message to be a str instead of a bytes;
the latter would surely fail later in the caller under python 3.X,
but the decode here probably only ever failed on python 2.X, where
a bytes result works because bytes is really just a synonym for str;
"""
try:
line = binline.decode(stream_encode) # [1.4] manual decode here, match subproc
except UnicodeDecodeError:
#line = b'(UNDECODABLE LINE): ' + binline # [1.6] don't let this kill the GUI in 2.X
line = '(UNDECODABLE LINE: see log file)\n' # [3.0] use str, but drop the content
return line
try:
binline = linequeue.get(block=False) # check the queue
except queue.Empty:
pass # nothing posted yet: reschedule and wait
else:
# process a batch of 1 or more objects
while True:
if binline != EOF_SENTINEL:
# process the next line string: GUI + logfile
eoln = os.linesep # local line-end: \r\n Windows, \n Unix
line = trydecode(binline, STREAM_ENCODE) # [1.4] manual decode here, match subproc [1.6]
line = line.replace(eoln, '\n') # [1.4] and fix any Windows eolns for tk
# [3.0] sanitize Unicode in line to be displayed in GUI
line = fixTkBMP(line)
# [3.0] allow comparison messages to be suppressed in the GUI, dynamically
anycompare = ('comparing', '"comparing', "'comparing") # ascii() in Windows exe!
if line.startswith(anycompare): # in line content?
if cmpmsgsvar.get(): # suppress toggle on?
if firstcompareline:
# show first line only for top-level dirs, plus message
firstcompareline = False
statustxt.insert(END, line)
statustxt.insert(END, 'Folder comparison messages '
'are being suppressed in the GUI...\n')
statustxt.see(END+'-2l') # scroll to new end of text
else:
# skip all other compare lines in GUI (only)
pass
else:
# show comparison line, reset to show message if toggled on+off
firstcompareline = True # reshow msg if toggled again
statustxt.insert(END, line) # add to end of text widget
statustxt.see(END+'-2l') # scroll to new end of text
else:
# show all other lines normally, don't reset for message
statustxt.insert(END, line) # add to end of text widget
statustxt.see(END+'-2l') # scroll to new end of text
# '-2l' = before empty auto \n at end
# force GUI to show/respond now (else dead during batch)
statustxt.update()
# write binary stream line to binary logfile
if logfile: # also save to log file? [1.4]: binary
eoln = os.linesep.encode() # must be bytes in 3.X (no-op in 2.X)
binline = binline.replace(b'\r\n', b'\n') # [1.4] got just \n from '-u' in 2.X only
binline = binline.replace(b'\n', eoln) # replaces are no-op in 3.X and unix
logfile.write(binline)
# read next line if any, else goto reschedule and wait
if linequeue.empty():
break
else:
binline = linequeue.get(block=False) # back to top of loop
else:
# reader thread posted eof sentinel and exited: close out the run
showinfo('%s: Finished' % APPNAME,
'The mergeall run has finished.' +
('' if not logfile else
('\nSee its log file in the popup window.' if logpopupvar.get() else
'\nSee its log file in the logs folder.')))
refocusWindow() # [3.0] else requires click on Mac
# if logging: close logfile, show in editor if toggled on [3.0]
if logfile:
logfile.close()
if logpopupvar.get():
# [3.0] Mac OS X is pickier about file URLs
if RunningOnMac:
logpath = 'file:' + logpath
#
# ANDROID [Apr1219] - webbrowser fails on Android (for reasons TBD),
# so spawn a shell command using the $BROWSER preset in Pydroid 3:
# "am start --user 0 -a android.intent.action.VIEW -d %s";
# safe to assume logpath is accessible (else can't write anyhow);
#
# ANDROID [Apr1919] - webbrowser _does_ work, but requires local file
# URLs to start with "file://"; prefix as required, and use either
# os.system or webbrowser.open to display the logfile in the GUI;
#
# ANDROID [Apr2119]: Pydroid 3 3.0 broke webbrowser and changed
# $BROWSER to skip "file://" - use os.system() + hardcoded command;
#
brw = 'am start --user 0 -a android.intent.action.VIEW -d %s'
url = 'file://' + logpath
cmd = brw % url
os.system(cmd) # _not_ os.environ['BROWSER'], webbrowser.open()
# other platforms code...
"""
# webbrowser opens text files in Notepad on Windows,
# gedit on Linux, and TextEdit on Mac OS X (but YMMV)
webbrowser.open(logpath) # assume never raises exc
"""
# [3.0] reenable new runs now
gobutton.config(state=NORMAL)
statustxt.config(state=DISABLED) # ANDROID [Mar2319]: back to readonly to avoid keyboard
# [3.0] the following had odd text scrolls after vertical resizes
# statustxt.pack_forget()
# gobutton.pack(side=BOTTOM) # unhide button
# statustxt.pack(side=TOP, expand=YES, fill=BOTH) # pack last=clip first
# exit the timer events loop: back to waiting on user
return
# end batch while loop
# reschedule and wait: check queue 10 times per second (msecs)
statustxt.after(100, streamconsumer, linequeue, logfile, logpath)
####################################################################################
# MAIN/TOP-LEVEL LOGIC
####################################################################################
if __name__ == '__main__':
if not RunningOnMac: # Windows and Linux: normal
root = Tk() # 'root' is used elsewhere
makewidgets(root)
root.mainloop()
else:
#----------------------------------------------------------------------------
# [3.0] Mac OS X hack: a partial workaround for a bug in the recommended AS
# Mac Tk 8.5 for Python 3.5. Hide/unhide main window so it shows its radio
# and check buttons in Aqua's active-window style (default blue) immediately.
# This code fixes style at initial opening only; style can be lost on popups
# and minimize/restore -- click this window, and possibly others first, to
# reset style as needed. More: docetc/miscnotes/mac-main-hack-notes.txt.
#
# This may be fixed in Tk 8.6, which may be supported by py.org's Python 3.X
# someday, and is supported by homebrew's Python distribution (to be tested).
# Unlike frigcal and pymailgui, a lift() is not enough here, whether clicked
# to open in the mac python launcher, or run from a 'python3' command line.
#
# Update: root.force_focus() is now used to restore window active state after
# dialogs, but doesn't help for deiconifies caught via <Map> or <Visibility>
# events, and has no impact here on initial state in all codings attempted.
#
# Update: the following don't help even if they do a focus_force (why?):
# root.bind('<Map>', onUnhide) + root.after(2000, refocusWindow),
# root.createcommand('::tk::mac::onShow', onShow),
# root.createcommand('::tk::mac::ReopenApplication', onReopen)
#
# UPDATE: losing focus in deiconifies in AS Tk 8.5 was finally fixed by the
# hideous workaround below, which creates and then immediately destroys a
# new but lowered (and hence invisible) top-level window on the Mac
# reopen-app even (i.e., Doc and app icon clicks). The widgets flash off
# and on momentarily (and the temp window may flash if mergeall is not at
# fullscreen), but otherwise are active styled. Tk 8.6 status tbd...
#----------------------------------------------------------------------------
root = Tk()
makewidgets(root)
# fix tk focus loss on startup
root.withdraw()
root.lift()
root.after_idle(root.deiconify)
# fix tk focus loss on deiconify
def onReopen():
#print(root.state()) # always normal
root.lift()
root.update()
temp = Toplevel()
temp.lower()
temp.destroy()
root.createcommand('::tk::mac::ReopenApplication', onReopen)
root.mainloop()
"""
# an alternative open workaround: a bogus Tk root, iconified after 2 seconds:
root = Tk()
root.protocol('WM_DELEsTE_WINDOW', lambda: None)
Label(root, text='Welcome to mergeall', width=25, height=5).pack()
makewidgets(Toplevel())
root.after(2000, root.iconify)
root.mainloop()
"""
"""
# a 15-line AppleScript alternative omitted here for space (and humanity...)
"""