File: thumbspage/examples/__prior-version-items/__prior-versions-code/viewer_thumbs-1.6.0-preCutDeps.py
#!/usr/bin/python3 """ ============================================================================== viewer_thumbs.py - part of the thumbspage image-gallery builder From: 'Programming Python, 4th Edition', Author/Copyright M. Lutz, 2010-2018. License: Provided freely but with no warranty of any kind. Display all images in a directory as thumbnail-image buttons that display full-size images when clicked. Requires Pillow for JPEGs and thumbnail image creation. To do: add scrolling if too many thumbs for window, and image scaling if image too big for display (see PyPhoto or thumbspage for both). This module is now part of thumbspage, which reuses the thumbnail generation and other functions here. This code has evolved much since its PP4E version, as described by the sections that follow. ------------------------------------------------------------------------------ 2017: Forked version in PyPhoto ------------------------------------------------------------------------------ PyGadgets' PyPhoto now uses a forked and enhanced version of this module, which can store all of a folder's thumbnails in a single pickle file, and better detects changes in the source images set: http://learning-python.com/ pygadgets-products/unzipped/_PyPhoto/PIL/viewer_thumbs.py ------------------------------------------------------------------------------ 2018: Integrate most PyPhoto fork enhancements ------------------------------------------------------------------------------ Modified to skip non-image items in image folder by MIME type, not exception, and integrate PyPhoto's fix for a Pillow tiff thumbs crash. Also uses latest "best" downsize filter name in Pillows that have it, and in script/GUI mode displays a note with PyPhoto/thumbspage links. PyPhoto's pickle file and change detection are still not used here. ------------------------------------------------------------------------------ 2018: Work around Pillow too-many-open-files bug ------------------------------------------------------------------------------ This module codes a workaround for a Pillow (PIL) bug that occurs only in limited usage contexts for folders having very many images. In brief, Pillow's auto-close of loaded image files does not work as documented, which can lead to "Too many open files" errors after many thumbnails have been generated. In thumbspage, this meant that results could reflect partial source content for very large folders, though only on Mac OS in general. The best fix is to manually open and close image files here, instead of passing filenames to Pillow. Read on for all the details. The Bug: This bug was seen only on Mac OS when running source code in a shell (specifically, on Mac OS 10.13 and 10.15, and Pillow 4.2.1 and 5.1.0). It does not occur when running a frozen Mac OS app, and does not occur when running source after shell limits are raised with "ulimit -n 9999". Mac app and shell limits can differ, but it's wildly convoluted (search the web for "mac os which command controls open file limits"). This bug has not been seen on Windows, in either command line or frozen executable contexts; Windows file limits are likely higher, in the system and/or C libraries. It may also vary by image type: on Macs it cropped up after processing 325 images in one use case, but fewer in others, and handles 470+ in one without error. You can read more about the bug on GitHub and such: https://www.google.com/search?q=pillow+too+many+open+files. This bug can also be triggered in PyPhoto, but only for very large folders; when running source code in a shell; on platforms with low open-file limits (Macs); and for runs that generate very many thumbs (unlike thumbspage, PyPhoto always caches and reuses thumbs made in prior runs). Frozen PyPhoto apps and executables are generally immune. By contrast, thumbspage is run via command line only, and must try to support all Pillows on all platforms. Subtle bit: opening the PyPhoto app from a shell via "open -a PyGadgets" on a Mac does not trigger the error; this bug is just a source-code issue (e.g., "python3 .../pygadgets/_PyPhoto/PIL/pyphoto.py -InitialFolder big"). It kills the GUI in PyPhoto, but can be avoided by a "ulimit -n 9999". PyPhoto note: learning-python.com/post-release-updates.html#pillowfilesbug. The Fix: The solution used here is to take control of the file away from Pillow altogether. Instead of passing a filename to Image.open(), manually open an image file; read it into a bytes, and pass it to Pillow wrapped in a BytesIO object; and manually close the file. This guarantees that files are closed. With this change, thumbspage supports folders of arbitrarily many images in all contexts (thousands work, albeit in a fairly large page). Note that the image.save() call does not leave files open. If it ever does, a similar fix may be used - instead of passing a filename, either pass a manually opened and closed output file, or a BytesIO object whose value is extracted and written to a file manually (set its .name, or use Pillow's EXTENSION[ext.lower()] for the image.save(obj) format argument). PyPhoto does similar for storing image data in a pickle file. Other Workaround Ideas: An explicit image.close() by itself is not enough, and makes the image object unusable. Although it also works to open with a temp image, .copy() to a work image, and explicitly .close() the temp image (as suggested on the web), this makes thumb builds much slower (~5X). This module could also simply not send the image object back in the result list (it's not used in thumbspage, and is auto-closed when reclaimed), but that won't work for programs that need to use the returned image objects (e.g., PyPhoto). ------------------------------------------------------------------------------ 2018: Auto-rotate images and thumbnails to display right-side up (optionally) ------------------------------------------------------------------------------ This module now automatically reorients (rotates) any tilted photos and their generated thumbnails to be right-side up (top on top), if they have valid "Orientation" Exif tags. This is especially useful for photos shot on smartphones, but applies only to images with Exif tags (JPEG and TIFF) from cameras and tools that tag files properly, and must change source-image files in place because web browsers will not rotate them when displayed in-page by the thumbspage client's viewer pages (see reorientImage() ahead). Because of its limits and changes, this feature automatically backs up image files with ".original" extensions, and both it and its backups can be disabled with new function arguments. If desired or needed, disable and manually rotate as preferred. Reorientation is not automatic or an easy option in Pillow (should it be?). For tests: examples/reorientation. ============================================================================== """ import os, sys, math import mimetypes, io # add for file type, buffer [2018] from tkinter import * # for GUI viewer mode from PIL import Image # <== required for thumbs (always) from PIL.ExifTags import TAGS # <== required for orientation tag [2018] #from PIL.ImageTk import PhotoImage # <== required for JPEG display (iff GUI) def printexc(): print('Exception:', sys.exc_info()[1]) def imageWideHigh(imgdir, imgname): """ -------------------------------------------------------------------------- Utility for thumbspage: use PIL to get image dimensions. -------------------------------------------------------------------------- """ imgpath = os.path.join(imgdir, imgname) imgobj = Image.open(imgpath) size = (imgobj.width, imgobj.height) # same as .size imgobj.close() # unless auto closed? return size def isImageFileName(filename): """ -------------------------------------------------------------------------- Borrowed from PyPhoto: skip non-image files explicitly. -------------------------------------------------------------------------- """ mimetype = mimetypes.guess_type(filename)[0] # (type?, encoding?) return mimetype != None and mimetype.split('/')[0] == 'image' # e.g., 'image/jpeg' def openImageSafely(imgpath): """ -------------------------------------------------------------------------- Work around a Pillow image-file auto-close bug, that can lead to too-many-files-open errors in some contexts (most commonly when running source code on Mac OS, due to its low #files ulimit). The fix is to simply take manual control of file opens and saves. For more details, see this module's top docstring. This does not appear to be required when opening existing thumb files, but is harmless. -------------------------------------------------------------------------- """ fileobj = open(imgpath, mode='rb') # was Image.open(imgpath) filedat = fileobj.read() fileobj.close() # force file to close now imgobj = Image.open(io.BytesIO(filedat)) # file bytes => pillow obj return imgobj def getExifTags(imgobj): """ -------------------------------------------------------------------------- Collect image-file metadata in a new dict, if any present (else empty). This stores each tag value present under its mnemonic string name, by mapping from image tag numeric ids to names, via the PIL.ExifTags table. -------------------------------------------------------------------------- """ tags = {} try: info = imgobj._getexif() # not all have Exif tags if info != None: for (tagid, value) in info.items(): decoded = TAGS.get(tagid, tagid) # map tag's id to name tags[decoded] = value # use id if not in table except Exception as E: pass return tags def reorientImage(imgobj, imgpath, backups=True): """ -------------------------------------------------------------------------- Rotate the image to be right-side up (top-side-top) if needed, before making its thumb. This also rotates the source image, and is simply an automatic alternative to manually rotating images before making thumbs. This might rotate for thumbs only and not save to source images, but not all viewers will rotate the source image when opened from the thumb. In this code's thumbspage client, specifically, browsers will not rotate the image when viewer pages display it in-page (PyPhoto may or may not need to save images too, but uses a forked/custom version of this code). Because this overwrites images, the original is saved to a ".original" copy if backups==True when rotated (which happens just once, after which no rotation is needed). Note: this uses transpose(), not rotate() - the latter may change more, subject to Pillow implementation and changes. -------------------------------------------------------------------------- """ tags = getExifTags(imgobj) # not all have Exif tags orientation = tags.get('Orientation') # not all have this tag if orientation: transforms = { # other settings don't make sense here 1: None, # top up (normal) 3: Image.ROTATE_180, # upside down 6: Image.ROTATE_270, # turned right 8: Image.ROTATE_90 # turned left } transform = transforms.get(orientation, None) # not all tag values used if transform != None: print('--Reorienting tilted image') imgstart = imgobj try: imgobj = imgobj.transpose(transform) # rotate to a new copy except: imgobj = imgstart # abandon the mission print('--Transform failed: skipped') # punt: can't auto fix printexc() else: # rotate source too, after backup? saveimage = True if backups: try: # be defensive: stuff fails backup = imgpath + '.original' os.replace(imgpath, backup) # rename + overwite prior except: # normally rotate just once saveimage = False imgobj = imgstart # abandon the mission print('--Source backup failed: skipped') printexc() if saveimage: try: imgobj.save(imgpath) # rotate source image too except: imgobj = imgstart # abandon the mission print('--Source rotate failed: skipped') printexc() return imgobj # new or original def makeThumbs(imgdir, # path to the folder of source-image files size=(100, 100), # (x, y) max size for generated thumbnails subdir='thumbs', # path to folder to store thumbnail files rotate=True, # reorient tilted images and their thumbs? backups=True): # save a .original of any images rotated? """ -------------------------------------------------------------------------- Get thumbnail images for all images in a directory. For each full image, either load and return an existing thumb, or create and save a new thumb. Makes thumb folder "subdir" in "imgdir" if it doesn't already exist. Returns a list of (image-filename, thumb-image-object), ordered only by underlying platform's directory-listing order: caller may wish to run the result through sorted() to unify ordering portably. Caller can also run os.listdir() on thumbs folder to access/load its thumbs after the call. On bad file types this may raise IOError, or other. Caveat: could also check file timestamps and source-image file existence to detect image changes and deletions requiring new thumbs (see PyPhoto, which does). As is, reusing existing thumbs for speed must assume no image changes. [2018]: Mod to use changes in PyPhoto fork: mimetypes, filter, tiffs. [2018]: Mod to avoid too-many-open-files bug in Pillow (see above). [2018]: Mod to auto-reorient images+thumbs right-side up if rotate==True. -------------------------------------------------------------------------- """ thumbdir = os.path.join(imgdir, subdir) if not os.path.exists(thumbdir): os.mkdir(thumbdir) thumbs = [] for imgfile in os.listdir(imgdir): if not isImageFileName(imgfile): # avoid exceptions [2018] print('Skipping:', imgfile) continue thumbpath = os.path.join(thumbdir, imgfile) # thumb name == image name if os.path.exists(thumbpath): # use already-created thumb (assume image same) thumbobj = openImageSafely(thumbpath) # Pillow workaround [2018] thumbs.append((imgfile, thumbobj)) # was Image.open(thumbpath) else: # make new thumb (image same or not) print('Making thumbnail:', thumbpath) imgpath = os.path.join(imgdir, imgfile) try: # work around Pillow auto-close bug [2018] imgobj = openImageSafely(imgpath) # was Image.open(imgpath) # rotate to right-side up, iff needed [2018] if rotate: imgobj = reorientImage(imgobj, imgpath, backups) if hasattr(Image, 'LANCZOS'): # best downsize filter [2018] imgobj.thumbnail(size, Image.LANCZOS) # but newer Pillows only else: imgobj.thumbnail(size, Image.ANTIALIAS) # original filter tiffs = ('.tif', '.tiff') if not thumbpath.lower().endswith(tiffs): imgobj.save(thumbpath) # type via ext or passed else: # work around C lib crash: see PyPhoto's code [2018] imgobj.save(thumbpath, compression='raw') thumbs.append((imgfile, imgobj)) # no need to discard image except Exception: # skip ctrl-c, not always IOError print('Failed:', imgpath) printexc() return thumbs # caller: use this or os.listdir(thumbs folder) #============================================================================= # GUI-mode code follows (used by __main__ and possibly other clients) #============================================================================= class ViewOne(Toplevel): """ -------------------------------------------------------------------------- Open a single image in a pop-up window when created. PhotoImage object must be saved: Tk images are erased if object is reclaimed. -------------------------------------------------------------------------- """ def __init__(self, imgdir, imgfile): Toplevel.__init__(self) self.title(imgfile) imgpath = os.path.join(imgdir, imgfile) imgobj = PhotoImage(file=imgpath) Label(self, image=imgobj).pack() print(imgpath, imgobj.width(), imgobj.height()) # size in pixels self.savephoto = imgobj # keep reference on me def viewer(imgdir, kind=Toplevel, cols=None): """ -------------------------------------------------------------------------- Make thumb-links window for an image directory: one thumb button per image. Use kind=Tk to show in main app window, or Frame container (pack). imgfile differs per loop: must save with a default. PhotoImage objs must be saved: erased if reclaimed. Packed row frames (versus grids, fixed-sizes, canvas). -------------------------------------------------------------------------- """ win = kind() win.title('Viewer: ' + imgdir) quit = Button(win, text='Quit', command=win.quit, bg='beige') # pack first quit.pack(fill=X, side=BOTTOM) # so clip last thumbs = makeThumbs(imgdir) if not cols: cols = int(math.ceil(math.sqrt(len(thumbs)))) # fixed or N x N savephotos = [] while thumbs: thumbsrow, thumbs = thumbs[:cols], thumbs[cols:] row = Frame(win) row.pack(fill=BOTH) for (imgfile, imgobj) in thumbsrow: photo = PhotoImage(imgobj) link = Button(row, image=photo) handler = lambda savefile=imgfile: ViewOne(imgdir, savefile) link.config(command=handler) link.pack(side=LEFT, expand=YES) savephotos.append(photo) return win, savephotos if __name__ == '__main__': """ -------------------------------------------------------------------------- Top-level script: make thumbs and show a simple tkinter image-viewer GUI. -------------------------------------------------------------------------- """ from PIL.ImageTk import PhotoImage # only here, to mimimize dependencies imgdir = (len(sys.argv) > 1 and sys.argv[1]) or 'images' main, save = viewer(imgdir, kind=Tk) # add other-viewers note [2018] popup = Toplevel() popup.title('Other Image Viewers') note = ('For better viewing options, see:\n\n' 'PyPhoto - at learning-python.com/pygadgets.html\n' 'thumbspage - at learning-python.com/thumbspage.html') label = Label(popup, bg='ivory', text=note, width=60, height=8) label.pack(anchor=CENTER, expand=YES, fill=BOTH) main.after(1000, popup.lift) main.mainloop()