""" ================================================================================ fixfrozenpaths.py: setup path/cwd context (part of the PyEdit [3.0]) In-place import-path and cwd setup for various distribution contexts: source, app, and executable. Setup differs per freeze tools for apps and executables. This file's code has been propagated to mergeall, frigcal, and PyMailGUI. Orignal notes: This code was initially added to allow PyEdit to be run standalone in the PyMailGUI package. When shipped with PyMailGUI's source-code release, the PP4E folder is in a custom location and not on PYTHONPATH: add its path here manually, so we can run PyEdit standalone too. We could reuse PyMailGUI's SharedNames.fixTkBMP() this way too, but that sets up too much coupling and dependency here - copy it. Later expansion: This has now grown more (really, wickedly) complex with the addition of app, executable, and standalone-source distributions. In these cases, the PP4E package is nested alongside this script or "baked in" to the frozen package, but these self-contained versions also need access to: Module: textConfig.py user configs, shipped separately in source form, everywhere. Located on sys.path when imported (and must be user-visible and editable). Data: The /icons folder for Tk program-window icons, on Windows and Linux. Located via cwd or dirname(__file__), in that order (client or own). Data: UserGuide.html, and icons+images in ./docetc that it uses, everywhere. Located via the dirname(__file__) directory, only. Data: The Help dialog also uses an image in ./docetc (like UserGuide.html), and later used the packages README file. Data: Auto-saves may rely on cwd for some save paths, including the config file's default of ./__pyedit-autosaves__, relative to the install dir. Script: The Capture Run-Code mode uses __file__ to locate its proxy script which may not be in "." in embedded-component mode (e.g., when used in PyMailGUI). For PyInstaller standalone executables (but not py2app bundles), the proxy script is also frozen and in '.', as sys.executable is PyEdit, not Python. See build-app-exe/windows/build.py for additional notes on this case. Mac apps currently use py2app, and Windows and Linux exes use PyInstaller. The build scripts or the installer commands they run (in build-app-exe) exclude the textConfig module so it is not frozen, and arrange for the above items to appear at accessible places: - ON WINDOWS AND LINUX, these items are copied by build scripts into the same folder as the single-file PyInstaller executable. This shipped folder shows up in dirname(sys.argv[0]), but not in dirname(__file__) which is always empty and maps to an arbitrary temp cwd (sys._MEIPASS) used to unzip the bundle. The executable's install folder is also not on sys.path automatically, and sys.executable is the exe, not Python. In fact, no Python executable is shipped, just its frozen library; each script must be a separately-frozen executable, with all its requirements embedded. For more backround, try: http://pythonhosted.org/PyInstaller/runtime-information.html. - ON MAC, these items are automatically included in the app bundle's folder Contents/Resources, to which py2app's Contents/MacOS bootstrapping code auto cds, and in which the main script (this file's importer) runs. This dir has both the main script and a zipfile with all PP4E package modules' bytecode. However, neither this folder nor '.' is on sys.path for imports automatically. sys.executable still refers to a real Python, whose executable is included in the app bundle, and which has all modules and packages named at build time "baked in." App scripts can thus be run as source with the app's Python. The sketchy docs are here: https://py2app.readthedocs.io/en/latest/index.html. In source-code packages, these items reside naturally in the source-code folder. Here, we need to set the cwd on some platforms for data, and must set the import path to see textConfig.py on all platforms, while taking care to allow for the portable/simpler source-code based distribution schemes. On Windows and Linux, the frozen subprocess proxy must do similar for textConfig imports, and must arrange for temporary folders to be pruned on non-normal exits. The cwd and sys.path are program-wide globals; __file__ is a module global, so changing it here would not impact this file's importer (not done). See also this file's __main__ code: Windows exes must patch multiprocessing, and Mac apps must catch Apple opendoc events manually (py2app argv emulation is broken, and PyInstaller fails for ActiveState Tk 8.5 or more). Running user code from a frozen app/exe is also problematic, and resolved by forcing all stdlib modules and allowing configs to name a Python: see subprocproxy.py. ---- FORMER CAVEAT: INSTALL PATHS AND UTILITIES For Windows and Linux PyInstaller exectuables, an os.chdir(exedir) was used in prior codings to make the empty __file__ dir map to the install dir in sys.argv[0] (instead of the _MEIPASS temp unzip folder). This allowed the __file__ to be used to portably derive the install folder. Unfortunately, this also precluded using relative paths in command-line arguments. This was not an issue in PyEdit or PyMailGUI (they have no or rarely-used args), but utility scripts in both frigcal and mergeall process file paths by nature. Omitting the chdir almost worked, but programs then could not find their icons, help, and spawned scripts if run from elsewhere via a command line instead of direct click. FIXED: the chdir is no longer run; instead, clients call fetchMyInstallDir() to select the install path explicitly per deployment mode -- from __file __ for source and Mac apps, and sys.argv[0] for PyInstaller exes. The cwd is unchanged, so relative paths work in all frozen executables' command lines. Exception: some programs (in frigcal) manually os.chdir() to the install dir returned from fetchMyInstallDir() early, for easy access to their data items relative to "."; this works only if no command-line path arguments are used. Here, the fix means the optional filename command-line argument can be relative to the folder where a command line is run. It's also possible to include some items in the unzip folder as Analysis data, but not items that must be user-visible -- including help and config modules. The new function also now allows for use in componnt mode: it expects a nested folder in the executable's dir for PyEdit data, only when running in component mode as a PyIstaller one-file executable (e.g., in PyMailGUI). ---- CONTEXT: WHY FREEZE? So why the bother? Apps and exes best support icons and desktop presence, and fully support associating pyedit to open text files automatically on clicks (.bat files work but flash on Windows, and Mac flatly requires an app bundle). Associations are especially important for a tool like PyEdit, which can open other files. They're made by simple right/ctrl-clicks on both Windows and Mac: - Mac: rightclick+Open With; or rightclick+Get Info+Change All for many - Windows: rightclick+Open With+Choose+Browse; or Control Panel+Defaults Linux, as common, is more complex (e.g., editing Ubuntu mime-list and .desktop files): https://duckduckgo.com/?q=associate+program+with+file+ubuntu Apps and exes also better support icons and drag-and-drop and right-clicks; do not require users to install Python too; and embed a specific Python version which makes future Python changes moot. Apps and exes can also be run by simple clicks instead of command lines (though this also works for source-code files on Windows, and via right-click to the auto-installed Python Launcher app on Mac). Further, on Mac apps can be drug to Launchpad (/Applications) for easy access. Downsides: source is demoted, and Mac apps badly obscure user config modules. The app/exe model also may not apply very well to decoupled/spawned scripts as in mergeall and PyMailGUI (the spawnee need not be an app, but can be frozen). Single-file executables on Windows also require a patch to be used with multiprocessing for Grep (see multiprocessing_exe_patch.py and __main__ for more details), pose unique hurdles for some subprocessing module use cases, and can convolute code substantially in non-trivial programs - per the next section. ---- POSTMORTEM FREEZE SUMMARY: A NET WIN? On final reflection, Python's frozen-executable support reflects clearly noble efforts, but seems chaotic, if not unrealistic, and significantly convolute an application's code. There are multiple very-different freezers; some have not been maintained as well as others; some fail completely on major platforms; some generate executables with very-long startup times or very-large folders; and most are riddled with special cases requiring wildly-hackish workarounds that can obscure the actual purpose of application code. Real-world tools like multiprocessing and subprocess likely won't work without coding heroics. Running arbitrary user code requires major help, due to the core model of minimal library inclusions. And the entire freezing paradigm is painfully similar to C development and makefiles, if not fundamentally at odds with both source-based scripting languages and Python's dynamic nature. In the end, PyEdit was able to make freezes work acceptably, and this does help much for icons, associations, and Python dependencies. Freezing effectively promotes scripts to first-class program status. But for non-trivial use cases this comes at a heavy cost in extra code convolution and development tasks, whose net you'll have to judge for yourself. This much is clear: placating unreasonable command-line phobia alone may be inadequate justification. For this developer's $0.02: py2app app bundles on the Mac seem more than justified, given their much-improved user experience on that platform. PyInstaller executables, though, seem to require too many code changes and bug workarounds to warrant the minor user-interface upgrades they provide, and other freeze tools seemed altogether orphaned or broken on Windows. On the other hand, insulating code from Python changes is an absolute win. ================================================================================ """ import sys, os DebugContext = False # trace before/after settings? # these are tedious to repeat RunningOnMac = sys.platform.startswith('darwin') RunningOnWindows = sys.platform.startswith('win') # or [:3] == 'win' RunningOnLinux = sys.platform.startswith('linux') #=============================================================================== def showRunContext(info=print, env=True): # info=(lambda *args: None) """ display this distribution's runtime context; for analyzing options and changes made here; caveat: __file__ is this mod's, not importer's; SEE ALSO: docetc/examples/RunCode/contextdump.py to display context in GUI spawned via Run Code; """ info('-'*25) info('PyEdit cwd: ', os.getcwd()) info('PyEdit __file__: ', __file__) info('PyEdit sys.argv[0]: ', sys.argv[0]) info('PyEdit sys.executable:', sys.executable) info('PyEdit sys.frozen: ', getattr(sys, 'frozen', '*notset*')) info('PyEdit sys.path:') for path in sys.path: info(' ', path) if env: info('PyEdit os.environ:') for key in sorted(os.environ): info(key, '=>\n\t', os.environ[key]) info('-'*25) #=============================================================================== # Set global import-path context if DebugContext: showRunContext() if hasattr(sys, 'frozen') and sys.frozen == 'macosx_app': # # 1) Mac py2app app-bundle folder distribution # Frozen importer's bootloader is in PyEdit.app's Content/MacOS dir. # Add '.' for importing config module in app's Content/Resources dir. # dirname(__file__) and cwd work for icons, UserGuide.html, scripts, # and imports: code is source files, run by the app's bundled Python. # Ok to use cwd here: py2app cds to the data dir by default anyhow. # assert RunningOnMac sys.path.append(os.getcwd()) # for textConfig.py import elif hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux): # # 2) Windows and Linux PyInstaller single-file executable distribution # Use exe's path (not temp _MEI*) for config module and all data items. # The config module cannot be in PyInstaller's auto-unzip Temp dir. # DROPPED os.chdir(): this made empty __file__ dir map to install dir, # but precluded any relative paths in cmdline args to frozen exes; see # fetchMyInstallDir() below for the later explicit-install-dir scheme. # exepath = sys.argv[0] exedir = os.path.dirname(os.path.abspath(exepath)) sys.path.append(exedir) # for textConfig.py import ##os.chdir(exedir) # for extras => now call fetchMyInstallDir! else: # # 3) Portable Source code distributions - nested within PP4E package or not: # - Run in self-contained PyEdit distribution with PP4E package nested here # - Imported or run in a client program, which nests PP4E, which nests me # - Imported by multiprocessing for grep producer in either PP4E scheme # Must use dir(__file__), not '.', to support relative pathname arguments. # srcdir = os.path.dirname(os.path.abspath(__file__)) if 'PP4E' in os.listdir(srcdir): # # 3a) Srcdir already on sys.path: no action required for imports # Data will be located by dirname(__file__) and/or cwd. # This might also check for the package in each sys.path dir. # pass else: # # 3b) Assume this file is nested in an enclosing PP4E package # Data will be located by dirname(__file__) and/or cwd. # Use __file__ to allow rel path args. Harmless if wrong. # upfromhere = os.path.join(srcdir, '..', '..', '..') sys.path.append(upfromhere) if DebugContext: showRunContext() #=============================================================================== def fetchMyInstallDir(__file__): # not global __file__ """ -------------------------------------------------------- call this to fetch folder where extra items reside; use to access installed icons, help, readme, scripts; replaces former os.chdir() which precluded rel paths; the return value is always an absolute pathname; pass importer's __file__ to __file__ arg: for frozen Mac apps, this module's dir(__file__) is in a zipfile, and differs from the importer's dir(__file__); they're the same for source code, and unused for Win/Lin exes; -------------------------------------------------------- """ if hasattr(sys, 'frozen') and (RunningOnWindows or RunningOnLinux): # # PyInstaller executable: from sys.argv[0] = exe's dir; # __file__ dir is empty and cwd may be any user folder # if the importer is run from a command line elsewhere; # # for pyedit only, data is in a component dir iff running # as a component in the pymailgui program: pyedit subdir # data is copied by build scripts to this nested folder; # exepath = sys.argv[0] exedir = os.path.dirname(os.path.abspath(exepath)) componentdir = '__pyedit-component-data__' if componentdir in os.listdir(exedir): return os.path.join(exedir, componentdir) else: return exedir else: # # Mac app bundle or source-code: from __file__ as usual; # cwd is anywhere: return importing file's install folder; # this mod's __file__ is *not* ok to use here for frozen # Mac apps: it's in a zipfile, not importer's Resources/; # srcpath = __file__ srcdir = os.path.dirname(os.path.abspath(srcpath)) return srcdir