File: pyedit-products/unzipped/

================================================================================ setup path/cwd context (part of the PyEdit [3.0])

In-place import-path and cwd setup for various distribution contexts: source, 
app, and executable.  Setup differs per freeze tools for apps and executables.
This file's code has been propagated to mergeall, frigcal, and PyMailGUI.

Orignal notes:

This code was initially added to allow PyEdit to be run standalone in the 
PyMailGUI package.  When shipped with PyMailGUI's source-code release, the 
PP4E folder is in a custom location and not on PYTHONPATH: add its path here
manually, so we can run PyEdit standalone too.  We could reuse PyMailGUI's 
SharedNames.fixTkBMP() this way too, but that sets up too much coupling and
dependency here - copy it.

Later expansion: 

This has now grown more (really, wickedly) complex with the addition of app,
executable, and standalone-source distributions.  In these cases, the PP4E
package is nested alongside this script or "baked in" to the frozen package, 
but these self-contained versions also need access to:

Module: user configs, shipped separately in source form, everywhere.
    Located on sys.path when imported (and must be user-visible and editable).
    The /icons folder for Tk program-window icons, on Windows and Linux.
    Located via cwd or dirname(__file__), in that order (client or own).
    UserGuide.html, and icons+images in ./docetc that it uses, everywhere.
    Located via the dirname(__file__) directory, only.
    The Help dialog also uses an image in ./docetc (like UserGuide.html),
    and later used the packages README file.
    Auto-saves may rely on cwd for some save paths, including the config
    file's default of ./__pyedit-autosaves__, relative to the install dir.
    The Capture Run-Code mode uses __file__ to locate its proxy script which
    may not be in "." in embedded-component mode (e.g., when used in PyMailGUI).
    For PyInstaller standalone executables (but not py2app bundles), the proxy
    script is also frozen and in '.', as sys.executable is PyEdit, not Python.
    See build-app-exe/windows/ for additional notes on this case.

Mac apps currently use py2app, and Windows and Linux exes use PyInstaller.
The build scripts or the installer commands they run (in build-app-exe)
exclude the textConfig module so it is not frozen, and arrange for the above
items to appear at accessible places:

- ON WINDOWS AND LINUX, these items are copied by build scripts into the same
  folder as the single-file PyInstaller executable.  This shipped folder shows
  up in dirname(sys.argv[0]), but not in dirname(__file__) which is always empty
  and maps to an arbitrary temp cwd (sys._MEIPASS) used to unzip the bundle.
  The executable's install folder is also not on sys.path automatically, and
  sys.executable is the exe, not Python.  In fact, no Python executable is
  shipped, just its frozen library; each script must be a separately-frozen
  executable, with all its requirements embedded.  For more backround, try:
- ON MAC, these items are automatically included in the app bundle's folder
  Contents/Resources, to which py2app's Contents/MacOS bootstrapping code auto
  cds, and in which the main script (this file's importer) runs.  This dir has
  both the main script and a zipfile with all PP4E package modules' bytecode.
  However, neither this folder nor '.' is on sys.path for imports automatically.
  sys.executable still refers to a real Python, whose executable is included
  in the app bundle, and which has all modules and packages named at build
  time "baked in."  App scripts can thus be run as source with the app's Python.
  The sketchy docs are here:
In source-code packages, these items reside naturally in the source-code folder.
Here, we need to set the cwd on some platforms for data, and must set the
import path to see on all platforms, while taking care to allow
for the portable/simpler source-code based distribution schemes.  On Windows
and Linux, the frozen subprocess proxy must do similar for textConfig imports,
and must arrange for temporary folders to be pruned on non-normal exits.

The cwd and sys.path are program-wide globals; __file__ is a module global,
so changing it here would not impact this file's importer (not done).

See also this file's __main__ code: Windows exes must patch multiprocessing,
and Mac apps must catch Apple opendoc events manually (py2app argv emulation
is broken, and PyInstaller fails for ActiveState Tk 8.5 or more).  Running
user code from a frozen app/exe is also problematic, and resolved by forcing
all stdlib modules and allowing configs to name a Python: see


For Windows and Linux PyInstaller exectuables, an os.chdir(exedir) was used  
in prior codings to make the empty __file__ dir map to the install dir in 
sys.argv[0] (instead of the _MEIPASS temp unzip folder).  This allowed the 
__file__ to be used to portably derive the install folder.  Unfortunately,
this also precluded using relative paths in command-line arguments.  

This was not an issue in PyEdit or PyMailGUI (they have no or rarely-used args),
but utility scripts in both frigcal and mergeall process file paths by nature.  
Omitting the chdir almost worked, but programs then could not find their icons,
help, and spawned scripts if run from elsewhere via a command line instead of 
direct click.

   FIXED: the chdir is no longer run; instead, clients call fetchMyInstallDir()
   to select the install path explicitly per deployment mode -- from __file __
   for source and Mac apps, and sys.argv[0] for PyInstaller exes.  The cwd is
   unchanged, so relative paths work in all frozen executables' command lines.
   Exception: some programs (in frigcal) manually os.chdir() to the install dir 
   returned from fetchMyInstallDir() early, for easy access to their data items
   relative to "."; this works only if no command-line path arguments are used.

   Here, the fix means the optional filename command-line argument can be 
   relative to the folder where a command line is run.  It's also possible 
   to include some items in the unzip folder as Analysis data, but not items
   that must be user-visible -- including help and config modules.

   The new function also now allows for use in componnt mode: it expects a
   nested folder in the executable's dir for PyEdit data, only when running
   in component mode as a PyIstaller one-file executable (e.g., in PyMailGUI).


So why the bother?  Apps and exes best support icons and desktop presence, and
fully support associating pyedit to open text files automatically on clicks
(.bat files work but flash on Windows, and Mac flatly requires an app bundle).

Associations are especially important for a tool like PyEdit, which can open
other files.  They're made by simple right/ctrl-clicks on both Windows and Mac:

  - Mac: rightclick+Open With; or rightclick+Get Info+Change All for many
  - Windows: rightclick+Open With+Choose+Browse; or Control Panel+Defaults

Linux, as common, is more complex (e.g., editing Ubuntu mime-list and .desktop 

Apps and exes also better support icons and drag-and-drop and right-clicks;
do not require users to install Python too; and embed a specific Python version
which makes future Python changes moot.  Apps and exes can also be run by simple
clicks instead of command lines (though this also works for source-code files on
Windows, and via right-click to the auto-installed Python Launcher app on Mac).
Further, on Mac apps can be drug to Launchpad (/Applications) for easy access.

Downsides: source is demoted, and Mac apps badly obscure user config modules.
The app/exe model also may not apply very well to decoupled/spawned scripts
as in mergeall and PyMailGUI (the spawnee need not be an app, but can be frozen).
Single-file executables on Windows also require a patch to be used with 
multiprocessing for Grep (see and __main__ for more 
details), pose unique hurdles for some subprocessing module use cases, and can
convolute code substantially in non-trivial programs - per the next section.


On final reflection, Python's frozen-executable support reflects clearly 
noble efforts, but seems chaotic, if not unrealistic, and significantly
convolute an application's code.

There are multiple very-different freezers; some have not been maintained as 
well as others; some fail completely on major platforms; some generate 
executables with very-long startup times or very-large folders; and most are
riddled with special cases requiring wildly-hackish workarounds that can 
obscure the actual purpose of application code.

Real-world tools like multiprocessing and subprocess likely won't work without 
coding heroics.  Running arbitrary user code requires major help, due to the 
core model of minimal library inclusions.  And the entire freezing paradigm is
painfully similar to C development and makefiles, if not fundamentally at 
odds with both source-based scripting languages and Python's dynamic nature.

In the end, PyEdit was able to make freezes work acceptably, and this does help
much for icons, associations, and Python dependencies.  Freezing effectively
promotes scripts to first-class program status.  But for non-trivial use cases
this comes at a heavy cost in extra code convolution and development tasks,
whose net you'll have to judge for yourself.  This much is clear: placating 
unreasonable command-line phobia alone may be inadequate justification.

For this developer's $0.02: py2app app bundles on the Mac seem more than
justified, given their much-improved user experience on that platform.
PyInstaller executables, though, seem to require too many code changes and
bug workarounds to warrant the minor user-interface upgrades they provide,
and other freeze tools seemed altogether orphaned or broken on Windows.
On the other hand, insulating code from Python changes is an absolute win.

import sys, os
DebugContext = False   # trace before/after settings?

# these are tedious to repeat
RunningOnMac     = sys.platform.startswith('darwin')
RunningOnWindows = sys.platform.startswith('win')           # or [:3] == 'win'
RunningOnLinux   = sys.platform.startswith('linux')


def showRunContext(info=print, env=True):          # info=(lambda *args: None)
    display this distribution's runtime context;
    for analyzing options and changes made here;
    caveat: __file__ is this mod's, not importer's;
    SEE ALSO: docetc/examples/RunCode/
    to display context in GUI spawned via Run Code; 
    info('PyEdit cwd:           ', os.getcwd())
    info('PyEdit __file__:      ', __file__)   
    info('PyEdit sys.argv[0]:   ', sys.argv[0])
    info('PyEdit sys.executable:', sys.executable)
    info('PyEdit sys.frozen:    ', getattr(sys, 'frozen', '*notset*'))
    info('PyEdit sys.path:')
    for path in sys.path:
        info('  ', path)
    if env:
        info('PyEdit os.environ:')
        for key in sorted(os.environ):
            info(key, '=>\n\t', os.environ[key])


# Set global import-path context

if DebugContext: showRunContext()

if hasattr(sys, 'frozen') and sys.frozen == 'macosx_app':
    # 1) Mac py2app app-bundle folder distribution
    # Frozen importer's bootloader is in's Content/MacOS dir.
    # Add '.' for importing config module in app's Content/Resources dir.
    # dirname(__file__) and cwd work for icons, UserGuide.html, scripts,
    # and imports: code is source files, run by the app's bundled Python.
    # Ok to use cwd here: py2app cds to the data dir by default anyhow. 
    assert RunningOnMac
    sys.path.append(os.getcwd())    # for import

elif hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
    # 2) Windows and Linux PyInstaller single-file executable distribution
    # Use exe's path (not temp _MEI*) for config module and all data items.
    # The config module cannot be in PyInstaller's auto-unzip Temp dir.
    # DROPPED os.chdir(): this made empty __file__ dir map to install dir,
    # but precluded any relative paths in cmdline args to frozen exes; see
    # fetchMyInstallDir() below for the later explicit-install-dir scheme.
    exepath = sys.argv[0]
    exedir  = os.path.dirname(os.path.abspath(exepath))
    sys.path.append(exedir)         # for import
    ##os.chdir(exedir)              # for extras => now call fetchMyInstallDir!

    # 3) Portable Source code distributions - nested within PP4E package or not:
    # - Run in self-contained PyEdit distribution with PP4E package nested here
    # - Imported or run in a client program, which nests PP4E, which nests me
    # - Imported by multiprocessing for grep producer in either PP4E scheme
    # Must use dir(__file__), not '.', to support relative pathname arguments.
    srcdir = os.path.dirname(os.path.abspath(__file__))
    if 'PP4E' in os.listdir(srcdir):
        # 3a) Srcdir already on sys.path: no action required for imports
        # Data will be located by dirname(__file__) and/or cwd.
        # This might also check for the package in each sys.path dir.
        # 3b) Assume this file is nested in an enclosing PP4E package
        # Data will be located by dirname(__file__) and/or cwd.
        # Use __file__ to allow rel path args.  Harmless if wrong.
        upfromhere = os.path.join(srcdir, '..', '..', '..')

if DebugContext: showRunContext()


def fetchMyInstallDir(__file__):     # not global __file__
    call this to fetch folder where extra items reside;
    use to access installed icons, help, readme, scripts;
    replaces former os.chdir() which precluded rel paths;
    the return value is always an absolute pathname;
    pass importer's __file__ to __file__ arg: for frozen
    Mac apps, this module's dir(__file__) is in a zipfile,
    and differs from the importer's dir(__file__); they're
    the same for source code, and unused for Win/Lin exes;
    if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux):
        # PyInstaller executable: from sys.argv[0] = exe's dir;
        # __file__ dir is empty and cwd may be any user folder
        # if the importer is run from a command line elsewhere;
        # for pyedit only, data is in a component dir iff running
        # as a component in the pymailgui program: pyedit subdir
        # data is copied by build scripts to this nested folder;
        exepath = sys.argv[0]
        exedir  = os.path.dirname(os.path.abspath(exepath))

        componentdir = '__pyedit-component-data__'
        if componentdir in os.listdir(exedir):
            return os.path.join(exedir, componentdir)
            return exedir

        # Mac app bundle or source-code: from __file__ as usual;
        # cwd is anywhere: return importing file's install folder;
        # this mod's __file__ is *not* ok to use here for frozen
        # Mac apps: it's in a zipfile, not importer's Resources/;
        srcpath = __file__
        srcdir  = os.path.dirname(os.path.abspath(srcpath))
        return srcdir

[Home page] Books Code Blog Python Author Train Find ©M.Lutz