![[Home page]](../PythonPowered.gif "Home page"){kind=small}
| Books | Code | Blog | Python | Author | Train | Find | ©M.Lutz |
File: genhtml/__docs__/prior-versions/1.0/genhtml--preSmarterInsertDependecies.py
#!/usr/bin/python3
"""
================================================================================
genhtml.py: static HTLM inserts
Author and copyright: M. Lutz, November, 2015
Simple static HTML inserts script: given an HTML templates dir and an HTML
inserts dir, generate final HTML files by applying text replacements, where
replacement keys and text are derived from names and content of insert files.
For static insert content, this script is an alternative to:
- Mass file edits on every common item change (which are painfully tedious).
- Client-side includes via embedded JavaScript (which not all visitors run).
- Server-side includes via PHP (which makes pages unviewable without a server).
This script does add a local admin step on every HTML content change, but there
is no true HTML include; <object> is close, but is not supported on (some?) IE.
Automatic change detection:
This script acts like a makefile - it automatically regenerates an HTML file if:
(a) the HTML template file has been changed since its last generation, or
(b) any inserts-dir file has been changed since an HTML file's last generation.
Usage pattern:
1) Change html-template files in SOURCEDIR (only!),
and/or insert-text files in INSERTSDIR.
2) Run this script to regenerate changed html files in TARGETDIR as needed.
3) Upload newly-generated files (or all) from the TARGETDIR to your web server.
In this mode, TARGETDIR is always a full mirror image of your web site.
Command usage:
% gensite.py (or click)
=> copy all changed non-html source files to target (if any)
=> regen and copy all html files that have changed since last gen
=> regen and copy all html files for which any insert file is newer
% gensite.py file.html+
=> same, but apply to just one or more files in source dir
Replacements:
- Replacement keys are all the 'XXX' in 'INSERTDIR/XXX.txt' filenames.
- Replacement values are the contents of the 'INSERTDIR/XXX.txt' files.
- Algorithm:
For each .htm and .html HTML file in SOURCEDIR:
For each 'XXX' in INSERTDIR/XXX.txt:
Replace _all_ '$XXX$' in HTML file with the content of file XXX.txt
Save result in TARGETDIR
Other non-HTML file types (if any) are copied to TARGETDIR unchanged.
To accommodate changing dates in both HTML files and insert files, and a regen
of every HTML file due to insert-file date changes, the script also:
- Replaces special '$_DATE*$' keys: '$_DATELONG$' => 'November 6, 2015'.
- Replaces any '$XXX$' keys in loaded insert-files text before any HTML.
The latter of these steps supports one-level NESTED inserts: insert files can
be built from other insert files, before being applied to HTML templates. For
an example use case, see FOOTER-COMMON.txt and its clients in the Html-inserts
folder. Note this step is only 1-level deep per file (it's not recursive!).
Example replacements:
$FOOTER$ => INSERTDIR/FOOTER.html (a standard sites links toolbar in <BODY>)
$HEADER$ => INSERTDIR/HEADER.html (a standard header block in <BODY>)
$STYLE$ => INSERTDIR/STYLE.html (a <style> or <link rel...> in <HEAD>)
$SCRIPT$ => INSERTDIR/SCRIPT.html (analytics JS code block in <HEAD>)
$ICON$ => INSERTDIR/ICON.html (site-specific icon link spec <HEAD>)
See also "__docs__/template-pattern.html: for a skeleton use case example
file, and the "Html-templates" test folder for additional template examples.
Miscellaneous notes:
- Skips replacement targets not present in HTML text (replace is a no-op).
- Skips replacement targets having no INSERTDIR file (no replace is run).
- Tries multiple Unicode encodings for HTML text if others fail.
- Blindly copies any changed non-HTML files in SOURCEDIR to target too,
but the primary and suggested usage mode is HTML files in SOURCEDIR only.
- To force a regeneration of all HTML files, open and save any insert file.
- Changed external CSS files must be uploaded, but do not require or trigger
HTML regeneration here (unlike changed CSS <link> or inline code inserts).
================================================================================
"""
import os, sys, shutil, time
INSERTDIR = 'Html-inserts' # insert text, filename gives key: XXX.txt.
SOURCEDIR = 'Html-templates' # load html templates (and others?) from here
TARGETDIR = 'Complete' # save expanded html files (and others?) to here
CLEANTARGET = False # empty target dir first if True
numcnv = numcpy = numskip = numfail = 0
# empty target dir?
if CLEANTARGET:
for filename in os.listdir(TARGETDIR):
os.remove(os.path.join(TARGETDIR, filename))
print('--Target dir cleaned')
# load insert files/keys and modtimes
inserts, insmodtimes = {}, []
for insertfilename in os.listdir(INSERTDIR):
insertkey = insertfilename[:-4] # key='$XXX$' from 'XXX.txt'
try:
path = os.path.join(INSERTDIR, insertfilename) # load insert text for key
inserts[insertkey] = open(path).read() # platform default encoding!
insmodtimes.append(os.path.getmtime(path)) # modtime for changes test
except:
inserts[insertkey] = '' # empty if file error
# add special non-file replacement keys (expand me)
inserts['_DATELONG'] = time.strftime('%B %d, %Y') # November 06, 2015
inserts['_DATESHORT'] = time.strftime('%b-%d-%Y') # Nov-06-2015
inserts['_DATENUM'] = time.strftime('%m/%d/%Y') # 11/06/2015
inserts['_DATETIME'] = time.asctime() # Fri Nov 6 10:44:58 2015
inserts['_DATEYEAR'] = time.strftime('%Y') # 2015
# replace any keys in loaded insert-file text first
for key1 in inserts: # for all insert texts
text = inserts[key1]
for key2 in inserts: # for all insert keys
rkey = '$%s$' % key2 # global replacement
text = text.replace(rkey, inserts[key2]) # no-op if no match
inserts[key1] = text
print('Will replace all:',
*('$%s$' % key for key in sorted(inserts)), sep='\n\t', end='\n\n')
# check run mode
if len(sys.argv) == 1:
filestoprocess = os.listdir(SOURCEDIR) # all files in source dir
else:
filestoprocess = sys.argv[1:] # or just filename(s) in args
# convert and/or copy files
def sourcenewer(pathfrom, pathto, allowance=2):
"""
was pathfrom changed since pathto was generated?
2 seconds granularity needed for FAT32: see mergeall
"""
if not os.path.exists(pathto):
return True # first gen
else:
fromtime = os.path.getmtime(pathfrom)
totime = os.path.getmtime(pathto)
return fromtime > (totime + allowance)
def insertsnewer(pathto, allowance=2):
"""
was any insert file changed since pathto was generated?
2 seconds granularity needed for FAT32: see mergeall
"""
if not os.path.exists(pathto):
return True # first gen
else:
totime = os.path.getmtime(pathto)
return any(instime > (totime + allowance) for instime in insmodtimes)
for filename in filestoprocess:
print('=>', filename, end=': ')
pathfrom = os.path.join(SOURCEDIR, filename)
pathto = os.path.join(TARGETDIR, filename)
if not filename.endswith(('.htm', '.html')): # tbd: skip subdirs?
# non-html (if any): don't attempt regen
if not sourcenewer(pathfrom, pathto):
# source file unchanged, don't copy over
print('unchanged, skipped')
numskip += 1
else:
# copy in binary mode verbatim and unchanged
rawbytes = open(pathfrom, mode='rb').read()
open(pathto, mode='wb').write(rawbytes)
shutil.copystat(pathfrom, pathto) # copy modtime too
print('copied unchanged')
numcpy += 1
else:
# html files: regen if html or inserts changed
if not (sourcenewer(pathfrom, pathto) or insertsnewer(pathto)):
# html unchanged and inserts not newer than result
print('unchanged, skipped')
numskip += 1
else:
# replace keys in text and copy over
for tryit in ('ascii', 'utf8', 'latin1'): # try unicode types
try:
file = open(pathfrom, mode='r', encoding=tryit)
text = file.read()
for key in inserts: # for filename/etc keys
rkey = '$%s$' % key # do global replacement
text = text.replace(rkey, inserts[key]) # no-op if no match
open(pathto, mode='w', encoding=tryit).write(text)
# no copystat(): need new modtime for insertsnewer()
except:
pass # try again with another encoding
else:
print('converted, using', tryit)
numcnv += 1
break # inner for loop, skip its else
else:
print('FAILED')
numfail += 1
summary = '\nDone: %d converted, %d copied, %d skipped, %d failed.'
print(summary % (numcnv, numcpy, numskip, numfail))
if sys.platform.startswith('win'):
input('Press enter to close.') # retain shell if clicked