|Version:||February 16, 2019|
|Author:||© M. Lutz, learning-python.com|
and view this package|
View this guide's media
View the related tkinter doc
This document describes how to use Mergeall to sync content on Android devices. It primarily focuses on running command lines in the Termux app, and more briefly explores a Mergeall GUI alternative on this platform. This coverage is provided freely, but with no warranties of any kind. Findings here reflect testing on Samsung Galaxy devices running Android 7 and 8 only. Given the rapid (really, supersonic) rate of change in the Android world, applicability to other devices and Androids cannot be guaranteed. Mergeall itself changes content by design: see its usage cautions before using the approaches outlined here.
If you're able to meet its requirements below, Mergeall on Android provides incremental backups and updates for content stored on your smartphone. With a few simple commands (or a fledgling GUI), you can sync content on your phone's internal storage or removable SD card both to and from an attached USB drive, and can do so right on your phone itself. The net effect turns your phone into a portable storage device for arbitrarily large content collections, without requiring card removal or device rooting, and without the cost, speed, and privacy negatives of cloud storage.
that hosts the guide you are reading is a supplement to the Mergeall system.
It comes with precoded helper scripts
that you can run directly on your phone from the Termux app's prompt. Though not
required, these scripts are designed to simplify Mergeall's command-lines mode on
Android, and minimize the risk of user errors. To make edits on smartphones easier,
these scripts also use common settings in the included
which you'll modify for your drives' IDs and folder names. Besides their role in
scripts, these settings provide quick command-line access to your folders in general.
For Mergeall users who break out in hives at the thought of using command lines, this guide also includes coverage of using Mergeall's GUI in the Pydroid 3 app, in its Appendix B. While this approach comes with code changes, operational issues, and pay-or-else advertising that make it a secondary choice here, some users may prefer a GUI even if it entails extra drama. If you wish to explore this option, be sure to read the Termux coverage in the main part of this guide first, as most of it applies to Pydroid 3 GUIs too.
Android's history and variability makes documentation challenging.
In this guide and its scripts, the terms "SD" and "USB" are
used to mean a removable microSD card, and anything that works on
the USB port, respectively, and the terms "external" and "removable"
are used interchangeably for a drive that can be detached from
your phone (i.e., SD or USB). If you don't have a removable card,
your phone instead has a nonremovable "external" partition on your
phone's internal drive set aside for storage—and confusion.
In the name of generality (and sanity), this package's scripts use the more function-related "data" to mean the content copy kept on your phone (e.g., on internal storage or removable SD card), and "plug" to refer to the content copy on your attached USB drive (e.g., on flashdrive or portable SSD). The term "phone" crops up here too, but this guide applies to Android devices of all types.
And to any grammarians in this document's audience: some of the closed compounds used here, including "logfile," "zipfile," and "fullscreen," may rankle a few purists, but they just make sense (the compounds, that is, not the purists).
The Mergeall system, originally developed for PCs, can be used unchanged on Android to synchronize on-phone content to and from a USB drive, without rooting your phone, and without requiring SD card removals—an arguably amazing feat in such a limited form factor. To make this work, though, you'll have to meet a few requirements up front. In short, Android has multiple issues and permission rules that require a custom approach; to use Mergeall on this platform, you:
The sections that follow go into these requirements in more detail. Later sections cover setup for these requirements, and usage of this package's helper scripts. The positive take here is that Mergeall on Android may be constrained, but it does work. While some users may prefer pulling their SD card and syncing on a PC, this is delicate work in the extreme, and isn't possible on all devices. With a USB drive, a little up-front work, and either simple command lines or GUI, Mergeall brings a more PC-like experience to your on-phone content management.
Mergeall works on Android devices, but only if they are running 2017's Android 8 (a.k.a. Oreo, and sometimes just O) or later. Earlier Androids have a showstopper bug which renders content-sync tools like Mergeall unusable. You can still use Mergeall to manage your phone's content prior to Oreo, but not on Android itself: remove your card and sync it on your PC.
The short story on the bug is that earlier Androids are unable to copy file modification times to either FAT32 and exFAT drives or internal storage on non-rooted phones: file content can be copied correctly, but copied files are always stamped with the time at which the copy was made, not that of the source. This is lethal to content-sync tools like Mergeall, which rely on file timestamps for comparison speed. Without timestamp copies, on-phone content copies cannot be compared to other copies accurately, and quick content syncs are impossible.
This issue is well-known, long-lived, device-wide, and not specific to
Mergeall, Python, or Termux. In fact, a shell
cp -p copy
command fails to copy times with an error before Oreo too—see the Nougat
It stems from Android's former FUSE-based filesystem, which required root
access to update file times but not content, and was eventually replaced
with Samsung's SDCardFS. For more background, try
the bug report
and searches like
As an example of its wide swath, this bug also kills calendar-file backup copies
in the Frigcal program, which makes a cameo appearance in
prior to Oreo, many programs were impossible on smartphones.
Though likely unintentional, it's unfortunate that the world's most-used mobile operating system didn't support non-cloud content syncing until 2017 (and still doesn't for many prior-version users). The good news is that this bug is now fixed, making cloud-free storage on Android finally practical for non-trivial content—and just in time for the larger-storage devices beginning to appear.
Due to a bug in the interpretation of exFAT file modification times on Samsung Android today, the alternative exFAT filesystem is not currently recommended for removable drives used with Mergeall on Android. In short, the modification times of files created on exFAT drives by Mac OS are skewed by 16 hours on Android, which can wreak havoc with Mergeall file comparisons.
You may be able to use exFAT anyhow: it works on most Androids today; the Android bug seems triggered only by exFAT files created on Mac OS; and drives and files used only on your phone may squeak by as incorrect-but-good-enough for Mergeall comparisons—files copied from exFAT drives may have skewed times, but so will times on exFAT drives. Until this bug is fixed, though, it's dire enough to recommend FAT32 as the better choice for your drives.
If you're willing to take that recommendation on faith, feel free to skip ahead to the next requirement now. For the curious and skeptical, though, the following traces the bug's forensics—beginning at the scene of the crime.
This bug is subtle, and it's not clear whether it lies in Samsung or Android software (and it's labeled "Android" here for brevity alone). But in brief: exFAT file timestamps created on some PC platforms are interpreted differently on Android than on any PC platform tested. That is, the same file on the same drive can report a different modification time on Android alone.
In other words, this appears to be a double bug: the Mac OS PC platform records exFAT timestamps in such a way that triggers an incorrect interpretation on the Android mobile platform. Files created on exFAT drives by Android itself have correct times everywhere; but those created on exFAT drives by Mac OS (only) report incorrect times on Android (only).
No, really. For a more visual look at the problem, here is a folder of files created on Mac OS High Sierra, as it's seen on Mac OS El Capitan, Windows 10, and Android Oreo ("Today" on Mac OS means 1/15/2019, and 1/16 was still tomorrow when this test was run). As you can see, the file times are wrong by 16 hours on Android alone—files created around 8:20 AM report their times near 12:20 AM the next day.
Conversely, files created on Android report the same and correct times on Mac OS, Windows, and Android, That makes this a one-way bug: it impacts just exFAT files created on Mac OS and viewed on Android, and not vice versa. You'd also never notice it if you stick to PCs (Windows times are correct on Mac OS and vice-versa), or limit your scope to just Windows and Android (Windows times are correct on Android and vice-versa).
The skew for Mac OS exFAT files on Android, though, makes them generally unusable across a platform mix that includes Android. These files' incorrect times may work on Android itself—Mergeall doesn't care that times are correct, only that they differ when files are changed (i.e., consistently incorrect is good enough). However, when Android copies files from an exFAT drive it will copy their incorrectly skewed times too. This makes the mistake permanent: if the copied files are ever copied unchanged from Android to PC, or the drive hosting such files on Android is ever used on a PC, the copied files will register as spurious differences and be recopied.
Happily, FAT32 drives are immune to this bug. To illustrate, here's what the same file saved on Mac OS looks like on Mac OS and Windows. On Android, this same file's time is skewed on an exFAT drive, but correct on a FAT32 drive. This means FAT32 provides a path for using Mergeall on Android. Less happily, this precludes mixed FAT32/exFAT usage: because only exFAT drives skew times, their files that hail from Mac OS won't compare correctly with the same files on FAT32 (or other) drives, and the same spurious differences and recopies will ensue.
Just as insidious, because an exFAT drive may have content created on multiple platforms, there is no automatic way to adjust the timestamps created on Mac OS alone. In content used across multiple platforms including a Macintosh, just some of your files, those made on Mac OS, will report incorrect times on Android. Programs cannot tell which differences are valid and which reflect Mac OS skew.
In sum, this Android/Mac OS double bug renders exFAT unusable on Android if your content includes—or may ever include—any files created on Mac OS. This is the only PC platform whose exFAT timestamps are known to trigger the timestamp bug on Android so far, so you might still be able to use exFAT drives for Mergeall on Android if you're sure that none of your content files will ever originate on a Mac. Still, it's impractical to test every PC platform in use (others may trigger the Android bug too), and you might not be able to foresee Mac OS-created files cropping up in the content you'll keep on Android on the future.
Because of this, it's impossible to recommend exFAT for Mergeall on Android today. Ultimately, this is a bug in Samsung or Android code that should be fixed: exFAT files report incorrect modification times on that platform alone, and files' sources should be irrelevant to this. Barring a future Samsung or Android fix—or a future Mac OS fix that skirts the Samsung or Android bug—you should use FAT32 for all the removable drives you will sync with Mergeall on Android. This is recommended for interoperability, and required for Mac OS content.
This is regrettable, given that exFAT supports larger files, and is otherwise more compatible with other filesystems' handling of times. Per their well-known limitations, FAT32 drives don't support files larger than 4G, and you may need to adjust your FAT32 drives' modification times manually twice a year to keep them in sync with non-FAT drives after daylight-savings-time rollovers. The latter category includes other drives that use UTC time instead of FAT's local time (and most PC internal drives do); see the FAT times fixer tool covered ahead for a simple way to make the adjustment.
The upside is that, thanks to Microsoft patent constraints, FAT32 is still much more widely usable today than exFAT. More importantly, its timestamps are implemented more consistently and interoperably; for Mergeall on Android, this is a killer asset.
The exFAT epilog: The web is woefully silent on this Android exFAT bug—which is hardly surprising, given that timestamps couldn't be copied on Android until 2017's Oreo. It's not impossible that Linux mount options may be involved (see this thread for leads), but it is impossible to use—or even explore—them on non-rooted phones. Another thread looks at a seemingly unrelated issue, but hints at pathology in Mac OS's exFAT implementation in general. The Mergeall development team has posted this bug on both Samsung and Android sites, but doesn't presently hold much hope for a rapid fix. Most support forums in the smartphone domain seem more likely to suggest restarting your phone and counting to 10.
On Android, Mergeall's Tk-based GUI works marginally with caveats, but its command-line mode works fully and qualifier free. To use this mode, you simply need to use Mergeall's source code, and type command lines to launch it. Although there are multiple ways to run Mergeall's Python source code on Android, the Termux app provides a full-featured Linux experience that seems best suited to the Mergeall task. Neither Termux nor command lines are required for Mergeall on Android, but they're suggested for most users.
In more detail, Mergeall's apps and executables for PCs (Mac OS, Windows, and Linux) do not apply to Android, and its reliance on the Tk library (and Python 3.X's tkinter interface to it) means that using its GUI's source code on this platform entails major tradeoffs. Although there is some new support for using Tk in Python programs run on Android today, it comes with rude advertising, substantial glitches, and extra steps that make it difficult to recommend for Mergeall syncs. See the notebox ahead for more on this story; it's evolving, but at this writing Tk may still be a nonstarter for many Mergeall users on Android.
Luckily, Mergeall is built with a decoupled architecture that allows its main code to be run separately from its GUI. To use Mergeall's command-line mode on your phone, you'll need to:
We'll walk through all these steps in more detail later in this guide. This is obviously more work than installing and opening a standalone app, but it's the easiest workable option for using Mergeall on Android today. On the bright side, this isn't nearly as difficult as it may sound. This package's helper scripts simplify the required command lines; the necessary installs are really just a few minutes of easy up-front work; and Termux provides a complete and general-purpose Linux command-line environment where Mergeall's source code is fully at home. As we'll learn later, a content sync is as simple as entering this at the Termux prompt:
This is roughly just a dozen taps, with the tab completion stressed repeatedly in this guide (and can be even faster if saved to a file). Although Mergeall's GUI comes with major downsides on Android today, its helper-script commands should still be easy enough to appease even the most chronically command-line phobic.
But if you really hate command lines: For an alternative approach, check out Appendix B's coverage of running Mergeall's GUI with the new tkinter support in the Pydroid 3 app's IDE. There, you'll learn more about using Python tkinter GUIs on Android, and sample a leading alternative to Termux for running Python code on this platform. Because this alternative comes with significant tradeoffs—and shares most of the requirements and setup tasks covered here for Termux—it's been relegated to the supplemental-reading section of this guide. Despite its rough edges, the GUI does work in Pydroid 3, and you may wind up preferring it (and this author may too); but this guide's Android topics still apply, and its suggestions are for a user base at large.
In addition to installing Termux, you must run its utility script
termux-setup-storage to create Termux's writeable app-specific
folders, and allow it to update files in internal storage and access removable
drives more broadly.
See this page
for more on the Termux setup command. Without it, Termux—and the
Python scripts like Mergeall that it runs—cannot read or update
most of the folders on your phone's internal storage or removable drives.
This makes content-processing programs like Mergeall unusable in Termux
until you run this command.
To illustrate the Termux permissions story, let's run some Python code within the Termux app. To do this yourself you'd need to install Termux and Python per ahead, but there's no reason to do so yet; this is just a demo. This also assumes some familiarity with pathnames (jump ahead to this section's notebox if you need more background); assumes basic Python knowledge (but you'll probably get the gist of this simple code either way); and uses Termux 0.65 (if your mileage varies in the future, it's probably due to a new Termux, a new Android, or both).
Before the Termux setup command is run, your storage options are severely
limited: on non-rooted phones, you can read and update files in the fully
private Termux app folder
/data/data/com.termux (the one holding
~ home), but not much more.
In Termux's Python, with internal
/sdcard, a removable drive named
/storage/8BB9-1202, and tracebacks trimmed for space here:
>>> open('/data/data/com.termux/xxx.txt', 'w').write('1334') 4 >>> open('/sdcard/xxx.txt', 'w').write('1334') PermissionError: [Errno 13] Permission denied: '/sdcard/xxx.txt' >>> >>> open('/storage/8BB9-1202/xxx.txt', 'w').write('1334') PermissionError: [Errno 13] Permission denied: '/storage/8BB9-1202/xxx.txt' >>> >>> open('/storage/8BB9-1202/Android/data/com.termux/xxx.txt', 'w').write('1334') FileNotFoundError: [Errno 2] No such file or directory: '/storage/8BB9-1202/Android/data/com.termux/xxx.txt'This code shows most writes (i.e., updates) failing, but read accesses at all these locations work the same way at this point. As you can see, there's not much accessible before the Termux setup command is run.
The Termux app-private folder at
/data is the standout exception.
It supports changes out of the box, but is largely useless for Mergeall, because this
folder cannot be accessed outside the Termux app on non-rooted devices. This
includes file explorers, photo viewers, and text editor apps, and disqualifies
this folder for most Mergeall roles. Because you really don't want to store your
content in a location that can be accessed by just one app (and a command-line-only
one at that), Termux's scope is very narrow when first installed.
Technically, at this point Termux can also read and update its nested
Android/data/com.termux app-specific folders like the one in the last
command above, provided you make them manually on either removal drives or internal
storage first. But you would have to do so in another app or on a PC: because
Termux cannot access the containing folders yet, it cannot build the required
folder paths under them. To do more, we have to move on.
After running the Termux setup command, your storage prospects improve radically. You—and hence Python scripts like Mergeall—have update access to all of internal storage, plus Termux's nested app-specific folders on removable drives:
>>> open('/data/data/com.termux/xxx.txt', 'w').write('1334') 4 >>> open('/sdcard/xxx.txt', 'w').write('1334') 4 >>> open('/storage/8BB9-1202/xxx.txt', 'w').write('1334') PermissionError: [Errno 13] Permission denied: '/storage/8BB9-1202/xxx.txt' >>> >>> open('/storage/8BB9-1202/Android/data/com.termux/xxx.txt', 'w').write('1334') 4
Read access for these folders is essentially the same as the write access shown, except that the failing folder here can be read without error too. That is, Termux, and the programs like Mergeall that it runs:
/data), though it's not very useful
/sdcard), though its space may be limited
/storage/xxxx-xxxx), like SD cards and USB drives
As noted, Termux's setup command creates its app-specific folders, like the one used in the last command run above, but you can create these folders manually too (and we will in the setup part of this guide). Running the setup command creates these nested app folders if they don't already exist; this ensures that Termux, and the scripts it runs, have a place to make changes on removable drives.
For a more comprehensive look at accessible paths which we'll skip in this guide, see this test script, and its Termux results; this Python updates session; and this before-and-after Termux session. More important here is the third command run above: even with enhanced permissions, Termux still cannot access all the folders on your removable drives—which leads to the next and perhaps strangest requirement.
For more on Android pathnames: By now, you may be feeling a bit confused by folder pathnames if you haven't done much with command lines in the past. For a gentler introduction to the subject, see this guide's Appendix A for extra background that we'll omit here. Like much technical documentation, this guide has to walk a thin line between alienating newcomers, and boring experts.
After you run the Termux permissions command of the prior section, Mergeall is able to both read and change content anywhere in internal storage, and can read content anywhere on an external (i.e., removable) drive. To allow Mergeall to make changes on external drives, though, your content must be nested in the drives' Termux app-specific folders. In Mergeall-speak, this nesting is not necessary if an external drive will only ever be the from copy in synchronizations, but it is required if the drive will ever play the role of to.
In more concrete terms, the Mergeall content folders that you will only read on your phone can be located anywhere, but those you will change (i.e., update) must reside at either of the following Android locations:
Android/data/com.termuxfolder of your external drives'
You can use the first of these for on-phone content copies if you have enough space, but must use the second for content on SD cards and USB drives that you wish to update. Since this is an arguably bizarre constraint, let's take a closer look at its rules.
First, the simple case: if you're lucky enough to have a phone with enough free space to store your content, internal storage is the easiest approach for on-phone storage, and recommended over SD cards. Internal storage:
Internal storage is also growing larger in recent phones, and is the only option
for on-phone storage if your device doesn't have a removable card.
To store your Mergeall content in internal storage, simply copy it to a
folder like the following on your phone;
/sdcard really means internal
storage, and the
MY-STUFF folder name here can be any name you wish:
This seems simple—and even PC-like—but internal storage isn't enough by itself. For one thing, even if your on-phone content is in internal storage, you'll still need to sync to and/or from an external USB drive in most use cases. For another, if your phone's available internal storage space isn't large enough for your content (and many today are not), you'll need to store your on-phone content on your device's SD card instead. Since both USB drives and SD cards require nested storage to support updates, let's turn to this model next.
Removable storage is generally slower, but it's required if your phone's internal space
is too small, and is still the ultimate in content control.
To store your changeable Mergeall content on external (i.e., removable) storage—including
USB drives and your phone's removable SD card—simply copy it to a
folder nested in the drive's Termux app-specific folder like this;
MY-STUFF means your stuff's folder:
Android uses IDs like this path's
25C9-1405 to uniquely identify
external drives. Your drives' IDs will naturally vary too, and are displayed in popular
Android file explorers (including
More unusual here: on both SD cards and USB drives, content folders like this path's
MY-STUFF must be nested six-levels deep in the drives' Termux app-specific
folder as shown, if you wish to update them with Mergeall on your phone.
This differs from internal storage, where anything under the
/sdcard root works.
Termux's explanation for why this is so is told on pages here and here. This may seem an odd requirement given that internal storage is wide open to changes, but it cannot be lifted in the context of Termux command lines today, and isn't as grievous in practice as it may sound. For instance, you can still access your content normally in a file explorer (as captured in this SD-card screen shot), and a home-screen shortcut (like this one) provides quick access to your nested content folder; see your file explorer to set one up.
Keep in mind that you can store content at the root of USB drives and SD cards too, but only if you'll only ever read—and never change—it with Mergeall. Unless you're sure that a removable drive's copy won't be the subject of a future Mergeall update, it should be nested in the drive's Termux folder. Also keep in mind that Android's permission rules have varied much and often in the past, and seem likely to do so again in the future. While some media may grow more accessible and some less, it's impossible to foresee tradeoffs in a platform that seems to change the rules every other Tuesday.
More on the (in)accessibility of removable-drive content:
This section noted that internal storage is recommended for storing content on your
phone because it is more accessible to other apps, but this merits a callout here.
Due to the permission rules we met earlier, some apps that
do not request broader access, including Termux, are unable to update content on
removable media except in their own app-specific folders. This is why you must
oddly nest your updateable content on removable media, including SD cards
and USB drives, to manage it with Mergeall in this app.
Unfortunately, this can also make it impossible to update such content in some other apps. As we'll see in Appendix B, the alternative Pydroid 3 GUI app has the same access limits—on removable media, it can update only in its own app-specific folder. For such apps, this is a bit of a catch-22: you must nest content on removable media to update it, but this cuts it off from other apps' updates. More tangibly, Pydroid 3 cannot update any content nested in Termux's app-specific folder on removable media—and vice versa. The net effect is that your choice of updating app is binding, unless you move your content.
Luckily, this caveat's scope is limited: it impacts only updates on removable drives, and does not apply to all apps. You can still freely view nested content on these drives, and modify it in general-purpose apps like file explorers and text editors that request broader permissions than Termux and Pydroid 3. If it's important to also update content in apps with narrower permissions, though, internal storage is your best option. Especially with the next section's peril factored in, the strange access limitations of removable drives on Android makes them less well-suited for single-folder content storage. Watch for a rehash of this topic when we consider Pydroid 3 at the end of this guide; it becomes crucial for alternative apps.
Finally, a bold-font caution: per normal Android operation (described by Android docs here and here), uninstalling Termux also silently deletes its app-specific folders on all your drives—including any content folders you've nested there. Unnested content copies, and unnested working folders in internal storage are immune to this, but those nested on your SD card and USB drives to support Mergeall updates may perish with Termux. Because this occurs without clear warning, it's substantially worse than the prior section's caveat on removable-media accessibility, and further bolsters the case for internal storage.
Specifically: on all devices tested, the
folders on both the removable SD card (at
and internal storage (at
/sdcard) were automatically deleted when
Termux was uninstalled, along with any and all content placed there by a user.
On one device, a file explorer also attempted to intervene and delete this
content too, but the system apparently beat it to the punch. There is
no reason to nest your content in an app-specific folder on internal storage—and
a root-level folder on internal storage avoids this issue altogether—but
there is no other option for updateable content on removable drives in Termux
(or other Python apps we'll meet in Appendix B).
This may sound scary, but it's designed to clear associated space when you're done
using an app, and really isn't much more dangerous than file-explorer deletions
if expected (and an
rm command in the Termux shell can be deadlier
still). Before uninstalling Termux, though, be careful to move your nested
content if you wish to retain it. In Android, uninstalls imply deletions.
And that's a wrap on the requirements front. In exchange for all this, Mergeall provides a free and proven incremental backup and propagation tool for the content you store on Android devices. Though not for everyone, this Mergeall approach requires neither device rooting nor card extraction—both of which can be perilous, if not impossible—and brings robust cloud-free storage management to your phone. If those benefits offset this section's requirements for your usage—or you just enjoy a technical challenge in general—read on for more on using this package's helper scripts to simplify Mergeall runs on Android.
Note from a possible future: If a standalone Mergeall app ever appears, it may obviate the need for both Termux command lines and content nesting in Termux's app-specific folders, but would still incur Android version and filesystem constraints. In particular, barring an exFAT fix, drives would still be best formatted as FAT32 for interoperability with Mac OS. Given the sloth with which bugs are fixed in commercial products like Android, using FAT32 today is likely not a short-term measure. If you're already using FAT32 drives, the only impact of an app upgrade would be an optional move of content to unnested drive roots.
Now that you've seen the what's required to use Mergeall on Android in general, this section shifts focus to the how. It provides detailed coverage of steps to follow to prepare your phone and drives for Mergeall use. Here's a quick rundown of these up-front tasks:
To be able to run Mergeall's Python scripts, you first need a command-line environment in which to run them. The Termux app provides a featured-rich Linux shell for Android that fits the bill; see its homepage for background. To get started, install the Termux app on your phone from the Google Play Store. Mergeall's command-line mode "just works" in Termux, because Android is really Linux under the hood.
Using Termux extra keys (or not):
Shortly after this guide was penned, Termux changed its extra-keys row
to be configurable, but in the process also removed keys crucial to
command-line editing (e.g., arrows).
To restore the former keys shown in this guide's Termux screenshots, visit
this page, and
scroll to its "Extra Keys Row" section. In brief: in Termux itself,
run a Unix
mkdir command to create the new folder
use a Unix editor like
nano to make a
termux.properties in the new folder and paste
in the line shown on that page; and restart Termux.
Option: some readers may
prefer to disable the Termux row entirely and install an alternative
on-screen keyboard that has all the important bits, like
Bonus: such a keyboard may also come in handy for tkinter GUIs run
on Android (stay tuned for the details in Appendix B).
To allow Termux to update both internal storage and its app-specific folders
on removable drives, launch the Termux app you just installed on your phone,
and run the following command at its prompt (the
$ in command listings
here is Termux's default prompt; type the text that follows it, and press whatever
means "enter" on your keyboard to submit the command):
After that, allow access in the
popup that appears.
Note that you may need to rerun this command later with removable drives
connected, if you get permission errors. This command also creates the
/Android/data/com.termux nested folders on all connected
media automatically if they do not exist, but you can also make these manually
when you're preparing your drives on a PC (as we will a few steps ahead).
If you've forgotten why this permissions step is required (or came here from a search), see the earlier coverage. Also note that Termux commands that update your phone's internal storage from here on in this guide assume that this command has been run; without it, such commands will fail on permission errors.
Mergeall is coded in the Python programming language. As mentioned earlier, we need to use its source code on Android because its PC executables don't apply, and source code requires a Python interpreter on any platform. To install Python for use in Termux, simply type the following command at the Termux app's prompt on your phone, and answer "y" (yes) when prompted:
$ pkg install python
This gets the latest Python 3.X (version 3.7 at this writing), which works
best for Mergeall. If you want to verify the install, type just
python at the Termux prompt:
$ python Python 3.7.1 (default, Oct 21 2018, 18:20:26) [Clang 7.0.2 (https://android.googlesource.com/toolchain/clang 003100370607242d on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys, os, platform >>> sys.platform, os.name, platform.machine() ('linux', 'posix', 'aarch64')
As usual, a
Ctrl+d key combination at Python's
prompt gets you back to Termux's
Now that you've got Termux and Python ready to go, it's time to setup a working folder on your phone where you'll run scripts to sync content. This is a simple task, but you'll want to decide up front where you wish to work, to avoid having to repeat steps later.
First, the recommendation: an
/sdcard/work folder located in the
root of your internal-storage space is an ideal candidate for your working folder,
because it's fast, accessible to file explorers and most other apps, changeable
by programs run in Termux, and short to type. This folder has the following multiple
aliases on devices used in testing, but shorter is generally better on a smartphone:
You can work in any writeable folder on your phone, but others come with tradeoffs that make them less than ideal in this role. For example:
~(which is really
/data/data/com.termux) is writable, but fully app-private: its lack of visibility in file explorers and other apps on non-rooted phones can make maintenance tasks like Bluetooth transfers harder.
/sdcard) might seem a natural too, but it's more to type, and will be automatically deleted if Termux is uninstalled (a broad issue discussed earlier).
For this guide, we'll assume and use the recommended working folder in internal
storage (and use its shorter
/sdcard name), because this is the
simplest and most functional location. To create your folder, run the
following in Termux on your phone:
$ mkdir /sdcard/work
You can also create this folder in your favorite Android file explorer—one of the benefits of locating it in internal storage this way. Once you've got your folder set to go, you're ready to populate it with software, per the next step on our list.
Wherever you choose to work, unzip (i.e., extract) both this helper-scripts package and Mergeall's source code there for convenience. The helper scripts simply augment Mergeall, so you'll need both packages. You can use the standard Mergeall source-code package, because its core syncing code developed for Mac OS, Windows, and Linux works unchanged on Android—a testament to the benefits of portable software (though the story for Mergeall's GUI in Appendix B won't be nearly as seamless).
For readers newer to Unix systems, the following is one way to install this helper-scripts package on your phone by running commands in Termux itself. First, in Termux, go to the working folder you created in the prior step:
$ cd /sdcard/workNext, fetch (i.e., download) this helper-script package and unzip it to your new working folder; click the links here on your phone to perform the fetch part of this recipe, then type the commands in Termux:
Fetch the zipfile at learning-python.com/mergeall-android-scripts.zip $ mv ../Download/mergeall-android-scripts.zip . $ unzip -d . mergeall-android-scripts.zip
Finally, do similar to install Mergeall's source-code package in the same working folder:
Fetch the zipfile at learning-python.com/mergeall-products/Mergeall-source.zip $ mv ../Download/Mergeall-source.zip . $ unzip -d . Mergeall-source.zip
For an example of the resulting working folder, see this
This assumes that clicking the links above on your phone makes the zipfiles
show up in
/sdcard/Download, but that's normal Android public-folder behavior.
Two tips here. First, if you're new to Unix, be sure to use Termux's arrow keys to recall prior commands to edit and resubmit, and its tab completion to let Termux fill in most of these commands for you. The latter makes command lines much easier to type; for instance, a tab after "m" in the sessions above likely expands it to "mergeall-android-scripts.zip," one after "M" fills in the Mergeall full zipfile's name, and "Dow"+tab expands to "Download" similarly. Don't type what you don't have to type—especially on a phone.
Second, there are other ways to achieve most of the above that we'll skip here.
For instance, you can fetch files on a PC and beam them to your phone; can use
Android file explorers instead of command lines for most steps in this process
(the former often call an unzip an extract, and refer to
. as the
current path); and can even download the zipfiles directly in Termux using
curl after a
pkg install curl:
$ cd /sdcard/work $ curl -O https://learning-python.com/mergeall-products/Mergeall-source.zip $ unzip -d . Mergeall-source.zip
Termux has the
wget download command too.
Regardless of their sizes, Linux-based devices offer an embarrassment of options.
Depending on the size of your content, the next step may take the longest. To use your content on Android, its storage structure must satisfy the media's filesystem and location requirements we met earlier here and here. In brief:
Since this is probably the most convoluted of the steps in this list (and is also required of the GUI alternative in Appendix B), the following sections provide a more detailed breakdown by media type.
Use your PC to setup the removable drives you'll use on Android.
First, you'll need to meet the FAT32 requirement. If your drive is already using
FAT32, you can move on to the next paragraph. Otherwise, start by copying any existing
Android content on your drive to a temporary location so it can be restored later;
this includes existing
DCIM (camera) and
Android (app-data) folders,
if present. Then, format your drive as FAT32, and restore the prior existing content you
saved. If you're new to formatting, see the notebox ahead for tips.
Next, place your content folder on your drive. Unless you will never update your drive's content on Android, its content copy must be placed in Termux's nested app-specific folder; here's how:
/Android/data/com.termuxfolder now if it does not exist, and copy your content folder there using whatever copy technique you normally use on your PC (e.g., file explorer or command line).
/Android/data/com.termuxfolder now if needed, and either move your content there if it's already located elsewhere on the drive, or copy it there anew.
For the last bullet above, you can move your content folder to the Termux app folder on the same drive with a single command on Unix-like PCs; here's an example on Mac OS (on Linux, use your media's paths; on Windows use drive letters instead of volume names, and move with DOS commands):
~$ mv /Volumes/EXT256-1/MY-STUFF /Volumes/EXT256-1/Android/data/com.termux/MY-STUFF
A caution here: you can move existing content in a file explorer too, but be sure a move in yours is a true move, and not a copy-and-delete. The speed difference can be staggering for large folders, and some Android file explorers may miss the mark on this.
Whether your content was copied or moved, it's now located in the Termux app folder—and four levels down from the drive's root—which can be inconvenient when coding paths and browsing for files. To make this easier to access on your PC, add a link (or shortcut) to it at your drive's root; here's how to do so on Mac OS with a symlink (that looks like this):
~$ ln -s /Volumes/EXT256-1/Android/data/com.termux/MY-STUFF /Volumes/EXT256-1/MY-STUFFThe link (or shortcut) is optional, but can save a lot of navigation time in file explorers, and typing in command lines. The root-level link probably won't work on Android, and you can't create one for removable drives on non-rooted phones because you don't have the required permissions on removable drives (this is the whole reason for nesting content in the first place); but it can save steps on PCs.
On Android, instead make a home-screen shortcut (that looks like
to your content folder; a longpress
in your file explorer is the usual first step. This makes more sense
for a resident SD card than a transient USB drive; but because your content is
now six levels deep when viewed on Android—nested in the
Android/data/com.termux folder of your drives'
/storage/xxxx-xxxx root—the shortcut can save a
whopping five taps per access, plus a file-explorer open.
If you're able (or required) to store your on-phone content in internal storage, you don't need to reformat as FAT32 or nest your content in the Termux app-specific folder. In fact, you shouldn't—internal storage is already formatted, and nested Termux app folders disappear on Termux uninstalls as noted earlier. Instead, simply create a folder for your content at the root of internal storage, and copy your content there. Both these steps must be done on your phone itself.
To begin, create your content folder in an Android file explorer, or use a Unix
mkdir command in Termux itself. For example, the following creates a
content folder named
MY-STUFF using a command in Termux; as usual,
your folder name may vary:
$ mkdir /sdcard/MY-STUFF
Then, copy your content to your phone. For most users, this will employ an intermediary USB drive: copy from PC to USB drive, then from USB drive to phone. A content copy on a USB drive you've already setup for Android in the prior section will suffice, and avoids another step.
A caution here too: on the phone, you can run the copy to internal storage in most Android file explorers, but some fail to copy over file modification times. At least one popular app, for instance, simply stamps every copied file with the current time, which is a showstopper for Mergeall—and no better than the pre-Oreo Android bug (in fact, this may be a lingering accommodation of it).
To make sure times make the move too, use Mergeall's
utility on your phone instead. There's more on this utility
in short, the copy on Termux looks like the following, assuming the settings we will
create in the next and final step are in place. That is,
you'll need to return here and run this after setting drive IDs in your bash profile file
(USB drive IDs are not available until that drive is prepped and attached,
and there's no way to show you this command without them):
$ cd $work $ py Mergeall-source/cpall.py $plug $data -skipcruft -vv -u > Admin-Mergeall/importlog.txt
Here too, once the content copy finishes, you may wish to create a home-screen
shortcut to its new location
on your phone. In this mode it's only two levels down in internal storage when
viewed on Android—at
/sdcard/MY-SYUFF—but the shortcut
still saves one tap per access, plus a file-explorer open.
A symlink doesn't make sense here, because it would be two levels down too (and
may be impossible in any event: a
ln -s at Termux's prompt didn't
/sdcard, despite having write permissions there).
A reminder: even if you use internal storage for on-phone content, you'll also need to setup a USB drive to propagate changes between your PC and phone (or save on-phone changes in general). That removable drive will still require FAT32 formatting for Mac OS interoperability and content nesting to support updates as described earlier, and can be used to initially populate your internal storage with content. Your on-phone content, though, is substantially easier to setup and use.
Tips on formatting your drives:
As noted, you'll generally need to reformat your
removable drives unless they already use FAT32, and this isn't a trivial task if you're
new to it. On Windows, the standard file Explorer won't format
larger drives as FAT32 (it forces exFAT instead), but a DOS
command and third-party options will; for pointers, try
this search and
On Mac OS, the Disc Utility app's Erase happily formats larger drives as FAT32 (if you select this and use uppercase drive names), and Linux users have many options. Android itself can format drives too (see your drive's Format in Settings ⇨ Device maintenance ⇨ Storage ⇨ Storage settings); unfortunately, it seems to silently choose FAT32 or exFAT for you based on drive size (a case of user-friendliness gone bad) so you'll have to use a PC for larger drives.
On all platforms, the usual warning about formatting applies: it erases your drive, so copy off anything valuable first. You may also want to add a
.nomedia file at the top of reformatted
removable drives that will be duplicates of the content kept on your phone
(e.g., on a go-between drive that will be used to propagate changes between your
phone and PC); where it works, this "hidden" file prevents Android's media scanner
from searching the redundant copies.
Last but not least, you'll now install and edit the bash profile file shipped in this package. This must happen last, because you may not know the Android names of your drives until they've been prepared and attached to your phone. Formatting, if run, changes the IDs Android uses to identify drives, and you'll need these IDs in this step.
Along with its scripts (and this guide), this helper-scripts package includes a
prototype bash profile file named
your--.bash_profile. This file has
simple settings that provide quick access to your Termux home and working folders,
as well as your content-copy directories on your phone and removable drives.
You can view this file's contents here;
it's mostly documentation (the lines that start with a
To use this file, delete the "your--" in its name; edit its path settings as both its
comments and the next section describe; and either save this file in your Termux home
folder or copy-paste its code into a
~/.bash_profile you already use.
Because your Termux home folder is app private, you will normally need to install the
profile file there using commands in Termux itself (not a file explorer). Here's the
required Unix incantation; this assumes you're
/sdcard/work and have already unzipped this package there:
$ cd ~ $ cp /sdcard/work/mergeall-android-scripts/your--.bash_profile . $ mv your--.bash_profile .bash_profile
For example usage of this file's settings at the Termux prompt, see this Android screen capture (and zoom in and zoom often). The bash profile file is also integral to using the helper scripts here: per the next and final setup section, it defines Android paths in a single location shared by all.
To make both commands and edits easier on smartphones, the helper scripts in this package use common content-copy path variables set in your bash profile file. This way, paths need be changed in that file only, and only when you first install this package or wish to use a new drive.
When you do need to change paths, the required edits are simple: in the
~/.bash_profile you created from the
prototype file, the settings to change are marked
as "EDIT ME." Replace these precoded paths' "xxxx-xxxx" IDs and literal
"MY-STUFF" with your own drive IDs and content-folder name, respectively.
Again, your drive IDs can be found in Android file explorers like
(make sure to match the case of drive ID letters exactly),
and folder name is whatever you wish it to be.
Here's one way to edit the profile file in Termux—because it's not
visible to other apps, you'll generally need to edit it in Termux itself too:
$ cd ~ $ vi .bash_profile $ source .bash_profile /sdcard/workThis runs
sourceto load your new settings after you're done, and uses the Unix
vitext editor. If
viis too cryptic for your tastes, the more user-friendly and WYSIWYG
nanotext editor also works in Termux after a
pkg install nano. You can also rename and edit the bash profile file on your PC and beam it over, but you'll still need to copy it into your home folder in Termux, because that folder is visible nowhere else.
Special cases: if you opt to use internal storage for your on-phone content copy
instead of SD card (per the requirements section earlier),
use the profile file's
/sdcard/MY-STUFF data-path setting instead.
You may need to also edit the scripts themselves to configure for your
Mergeall source-code package's install location if it's not in
(the current working directory), but this is not required if Mergeall's package
is unzipped to the same folder as this scripts package.
Because removable drives are identified by drive IDs in Android, the paths
used by helper scripts will change if a new removable drive is inserted. To reconfigure
for a new drive, simply update paths in your
~/.bash_profile with the editing
commands above; all Android helper scripts automatically
source (i.e., reload)
this file on startup, so they always use its current settings.
You may also need to rerun Termux's
termux-setup-storage when changing
to a new drive; if you get permission errors, this applies to you.
Helper-scripts coding notes:
The symlinks made at drive roots by the Mac OS move-and-link
script here might suffice for pathnames in other
scripts, but these links won't generally work on Android, and non-rooted phones don't have
permission to make such links in removable drives on Android itself.
~/storage/external-* links also record drive
paths, but they seem too ambiguous to use: they give IDs but not drive type, and are
not always current.
Instead, the helper scripts rely on the manual
settings in your bash profile file, both for generality, and because explicit is
usually better than implicit—especially when your content is at stake.
diffall.pybyte-for-byte comparison utility
Additional scripts run modification time (a.k.a. modtime) adjustment tools for FAT32
drives, and other Mergeall utilities are available as standard Unix command lines.
As a bonus, all the Android helper scripts here automatically save logfile output to
your working folder's
Admin-Mergeall subfolder, using filenames either
passed or generated; to see how this works, let's jump right into the available commands.
To sync or compare your content: select the Mergeall helper script
in this package whose name matches your goal; open the Termux app on your phone;
and run the selected script with a
bash scriptname.sh command
line at the Termux prompt. For example, the following launches a USB-to-phone
updates run, assuming this package has been unzipped in your working folder:
$ bash mergeall-android-scripts/mergeall-plug-to-data-update.sh
bash command just tells Termux to run all the precoded commands
in the chosen file, and scripts' filenames imply their action. For instance,
plug-to-data in this script's name syncs changes from USB drive to on-phone storage,
update means apply changes to the destination automatically;
data-to-plug syncs the other way, and
report means report
differences only without changing anything.
Worth reiterating: be sure to use shell tab completion to fill in the folder and script
names in these commands; in this case "bash m+tab/m+tab" is most of the command.
Optional: scripts located in your Termux home folder (
also be made more directly executable with a
chmod +x scriptname.sh
and run with just a
scriptname.sh; in Termux this works for
home-folder scripts only, not for working folders located
Mergeall's scripts—and hence this package's bash helper scripts that run them—both
process files and generate logfiles that detail their actions and results. On Android, script
logfile output is automatically saved in your working folder, to subfolder
$work/Android-Mergeall, in a file whose name is either:
bash scriptname.sh filename
Whether the logfile name is passed or generated, the logfiles subfolder is created automatically if needed.
You can also launch scripts with
& at the end of the bash command
bg after it starts) to run in the background,
so you can monitor output with a
tail -f logfile.txt. For instance,
the following session ensures that Termux is in your working directory by running the
gowork command defined in
~/.bash_profile; starts a USB-to-phone
update run, saving its output using a generated filename; and monitors script progress:
$ gowork $ bash mergeall-android-scripts/mergeall-plug-to-data-update.sh & $ tail -f Admin-Mergeall/mergeall-somedate-sometime.txt output scrolls here Ctrl+c to exit
Alternatively, the following reports file differences only, saves output to
an explicitly named file, and views output only after script completion
vi text editor (
nano also works after
an install as noted earlier, you can switch to another Termux session while
the merge runs, and you don't really need to
gowork again unless
you've gone elsewhere):
$ gowork $ bash mergeall-android-scripts/mergeall-plug-to-data-report.sh ma.txt $ vi Admin-Mergeall/ma.txt
Besides its main merge script, Mergeall includes the
deep-comparison utility, and the helper scripts include a script to launch it.
The following, for example, compares content copies byte for byte on Android,
monitoring output as it appears in a passed-in filename; there is no
from and to ordering for drives here, because this isn't a merge:
$ gowork $ bash mergeall-android-scripts/diffall-plug-and-data.sh diffs.txt & $ tail -f Admin-Mergeall/diffs.txt output scrolls here Ctrl+c to exit
As a guideline, you should run this comparison script occasionally to ensure that all files in your content copies are completely identical. Though Mergeall's time-and-size comparisons are fast and adequate for your day-to-day syncs, drives can and do go bad eventually. A byte-for-byte check proves the integrity of your content—albeit much more slowly (see the notebox ahead).
Additional helper scripts in this package run Mergeall's
to adjust file modtimes on FAT32 drives so they are in sync with non-FAT drives
after daylight-savings-time changes. Run the helper script like this, with an
-sub followed by a number-of-hours value,
and with or without a manual logfile name:
$ gowork $ bash fix-fat-dst-modtimes-data.sh -add 1 logfile.txt $ vi Admin-Mergeall/logfile.txt
Because you must use FAT32 for broadest interoperability on Android today, you may need to need to run this utility to synchronize with other types of drives twice a year. Otherwise, FAT32's local times may all be one hour off from those on drives that use UTC times (a Mergeall report-only run with reams of differences is this state's symptom, and the times of a file on both drives is its proof). See Mergeall's user guide and the fixer script itself for more details.
For a more graphic look at the Mergeall helper scripts in action
on Android, check out the Mergeall-run screenshots
and the exemplary but heavily redacted
And for a comprehensive test that verifies the approaches laid out in this document,
study the files in this folder,
especially its test notes.
Though most phones would lose a straight-up footrace
with a PC, on Android, Mergeall itself is relatively fast because it compares just
file timestamps and sizes by design. The more exhaustive
utility, though, can run much longer for large content collections—which is
one reason for using incremental sync tools like Mergeall (though even-slower
full copies make it a slam dunk).
As an example, on-phone comparisons of a 150G content collection having 99k files and 9K folders on microSD and USB flashdrive work well, but take two hours in
diffall.py versus two minutes or less in Mergeall.
For long-running tasks like the former, be sure to acquire the Termux partial
wake lock in
Android's notifications to keep your program running even after your screen
Remember to release it when your task finishes to avoid battery drain.
And for true sports fans: the Mergeall comparison for the same 150G content folder on the same drives clocks in at 24 and 54 seconds on Mac OS El Capitan and Windows 7 machines, respectively—though the variables are many, and merges still beat microSD-ectomies.
Mergeall's source-code package also includes the
script, and its
test subfolder includes the ziptools package which
includes create/extract scripts. If you elect to host your content on your phone's
internal storage instead of a removable card, for example, you might use either of
these to copy your content folder in full from a USB drive initially, or export
it to USB later.
Though not wrapped by helper scripts here, both of these tools can be run in your
$work folder with normal Unix command lines in Termux, using either
your bash-profile settings or custom paths. For instance, both of the following
command pairs import and export content copies on your phone ("py" is assumed to
be aliased to "python" in your bash profile file here, and routing the logfile is
up to you in this mode):
$ py Mergeall-source/cpall.py $plug $data -skipcruft -vv -u > Admin-Mergeall/importlog.txt & $ py Mergeall-source/cpall.py $data $plug -skipcruft -vv -u > Admin-Mergeall/exportlog.txt & $ py Mergeall-source/test/ziptools/zip-extract.py $plug/../import.zip $data $ py Mergeall-source/test/ziptools/zip-create.py $plug/../export.zip $data/* -skipcruft
As usual, tab completion helps with input here, and bash scripts, variables, or aliases
can shorten such commands.
For more on using
cpall.py, see the
for more on ziptools, see its homepage.
Termux itself comes with a Unix
unzip and a
pkg install zip adds a Unix
zip; both generally work
well, but are naturally subject to the Unix tools' constraints (e.g., file-size
limits may apply).
If you prefer a more interactive experience, Mergeall's console mode works on Android too. This mode asks you for input parameters, instead of expecting them in the command line. It doesn't have tab completion for paths, though, so it's a bit more typing or pasting, and may work better with a Bluetooth keyboard. See the screenshots of this mode's setup and results, and start it in Termux with a command like this:
$ py Mergeall-source/launch-mergeall-Console.py
Finally, you can also skip the helper scripts and bash profile altogether and run Mergeall's scripts directly, as described in its user guide. The following Unix command line, for instance, kicks off a Mergeall USB-to-SD updates run on Android manually:
$ python Mergeall-source/mergeall.py \ /storage/7284-2735/Android/data/com.termux/MY-STUFF \ /storage/25C9-1405/Android/data/com.termux/MY-STUFF \ -auto -backup -quiet -skipcruft \ > ../Admin-Mergeall/mergeall-$(date +%Y%m%d-%H%M%S).txt &
And if you can imagine typing this all on a smartphone's touchscreen, you'll probably appreciate why the helper scripts may be a better idea.
This section collects assorted usage pointers and reminders. Most are intended for users newer to Termux, Unix, or both.
emacsin Termux, or text-editor apps
nanois user friendly: run
pkg install nanoto get, and tap
Alt+xfor more space
pythoninstall is Python 3.7 today; install
python2for 2.X code (info)
pkg install Xinstalls Termux tools;
pip install Xinstalls Python tools in Termux
pip install Pillow, after running this command (but without its "-y")
In closing, if this Mergeall approach seems like a lot of work, it's because it is, but it's also a price inherent in maintaining content on Android smartphones with Termux command lines today. In exchange, this scheme provides a fully functional incremental-backup and changes-propagation tool that you can run directly on your Android device to synchronize with external drives—no device "rooting" or card juggling required.
That being said, some users of both Mergeall and Android may prefer to remove cards instead of using command lines, and this may be the only option for others. Still, any scheme that can free your content from the constraints of both closed devices and cloud storage is worth a look. Not only are storage devices much quicker than clouds, they can automatically neutralize the proprietary and even exploitive agendas that seem to underlie much of today's smartphone experience.
However you choose to proceed, taking control of your digital property is worth the effort.
This appendix introduces and summarizes the folders used in this guide. It's probably more useful to readers who haven't used command lines much in the past, but also serves as a glossary of sorts.
Because the locations at which you'll store your content on Android are central to the approach presented here, it's important to get a handle on these up front. Android identifies a storage location just like every other platform, with a pathname—a string of folder names that reflects folder nesting. Some of Android's naming conventions are unique, though, and some pathnames identify folders with specific roles. Here's a quick rundown on those that play a part in the Mergeall story:
This folder is really located in internal storage, but is too hidden
to be accessed as such. Being Linux-based, Android uses the
character to separate folders in a pathname like this; the one all the way to
the left means the device root (the top of the folder-nesting tree),
and the left-to-right order of
/-separated names gives their nesting.
This is where you normally "are" when you first open Termux (before
you run a
cd to go to a different folder).
It also goes by the names
which are just shell syntax that automatically equate to this folder's
full pathname above.
Because you can't see this folder outside the Termux app, its scope
is limited. Because of the way Termux works, though, this is where you'll
need to install and edit the bash profile file covered in this guide.
/sdcard /storage/emulated/0And yes,
/sdcardreally means internal storage, not your removable SD card, for historical—and even tortuous—reasons we'll omit here. Because internal storage is accessible to every app, it's ideal for content storage if your phone has enough space. It's also generally more convenient than your "home" folder for working in Termux, because you're not limited to Termux tools. If you store your content and working folders in internal storage, their pathnames would look like the following (your stuff's
MY-STUFFmay naturally vary):
/storageroot on Android when they are attached, with a folder name derived from the drive's unique ID. For example, here's one for a USB drive with an ID of
/storage/25C9-1405Your drives' IDs will differ (they're supposed to be unique, after all), and you can find them in some Android file explorers after drives have been attached (and the system has a chance to mount it in the folder-nesting tree); see earlier for app pointers. A physical drive's ID will change if it's reformatted, but should otherwise remain constant.
Every app has one of these folders if it needs one; they also appear in
internal storage, and are meant for app-specific content that might be too
large or inconvenient to store in the
/data app-private folder.
As discussed in this guide, these app-specific folders are key to using
content on removable drives in Mergeall—whether from Termux command
lines, or the GUI of the next appendix.
App-specific folders like these are also automatically and implicitly deleted when the owning app is uninstalled. This is meant to clean up an app's resources, but makes these folders subpar for storing data. Because of that, you'll generally locate your working folder elsewhere. Your Mergeall content would ideally be elsewhere too, but updatable copies on limited-space phones may have no other option than removable storage, and this requires app-specific folders today.
Keep in mind that all of the above reflects Android's conventions. If you use the same removable drive across different platforms, the names of its folders will differ on each. The following, for example, is the pathname for the same folder on Android, Mac OS, Linux, and Windows; all reference the same folder, but you'll need to extrapolate from the Android flavor shown in this guide when using a drive elsewhere:
Android: /storage/25C9-1405/Android/data/com.termux/MY-STUFF Mac OS: /Volumes/EXT256-1/Android/data/com.termux/MY-STUFF Linux: /media/username/EXT256-1/Android/data/com.termux/MY-STUFF Windows: D:\Android\data\com.termux\MY-STUFF
Also keep in mind that the pathname forms above and used in this guide
may vary on some Android devices, and have varied over Android's history.
worked in the past too. The paths used here reflect Samsung Galaxy
devices using Android Oreo and Nougat, so you'll have to translate if
those on your phone differ. Alas, Mergeall's budget is limited, and
stability is not among Android's core virtues.
For more details on Android pathnames, see books, the web, or other Android resources.
The main part of this guide recommends and covers running Mergeall by using command lines in the Termux app, because this approach is both straightforward and reliable, and is completely advertising and payment free. Termux provides a free, full-featured. and general-purpose Linux environment, which can run Mergeall's command-line mode with minimal setup tasks and robust performance. This is not, however, the only Python-on-Android game in town.
Most notably, the Pydroid 3 app has recently acquired some rudimentary though impressive support for using the Tk GUI library—and Python 3.X's tkinter interface to it—in Python programs. This makes it possible to use Mergeall's GUI on Android devices today, along with a host of other existing tkinter software. Less positively, this app's tkinter support also comes with substantial downsides, including both behavioral glitches and unfortunate advertising, that make this approach difficult to recommend in this guide.
That being said, the prospect of running PC GUIs on a smartphone is arguably seductive, and the possibilities that this app raises for Mergeall users merit both fair review and your own judgement. You may or may not prefer Pydroid 3 GUIs to Termux command lines in the end, but this appendix provides enough coverage of the former to help you make a good usage-mode choice.
Versions note: The findings in this appendix reflect Pydroid 3 version 2.22, Python version 3.6, and Android versions 7 and 8 (Nougat and Oreo), and all three play roles in issues noted along the way. While this how-to hopes to be revised for fixes in these systems, change is constant and rapid in the software world. If you're reading this in the future, be sure to check all three for current status. With any luck, the glitches list here will be completely invalid by the time you see it.
First and foremost, Mergeall's GUI does work in Pydroid 3 with only minor changes, and without the command lines required for Termux use. Under this alternative approach, your content syncs will be run in the Pydroid 3 app's IDE (its edit+run GUI) that looks like this, and the Mergeall GUI you will use to configure and run syncs on your phone will look as follows:
Despite the tradeoffs we'll explore in this appendix, it's important to note that running tkinter programs this way is an enormous achievement, given the large disparities between PC and Android platforms.
Now that you've seen Mergeall's GUI at work, we need to define some terms that impact the way it renders in Pydroid 3. The first two sets of screenshots above were captured with this app's tkinter fullscreen and maximized viewing modes enabled, by setting the first Tkinter switch off and the second on in the app's Settings ⇨ System dialog. You can use neither, either, or both of these modes, but their utility can vary per GUI and usage:
Nevertheless, Mergeall seems to work best with both these modes enabled, because it has no persistent popup windows and doesn't need to catch program exits, and because screen space is a precious commodity on phones in general. Mergeall works without these modes too—and looks like the last set of screenshots above—but has less space, and may bear manual resizing which seems out of place on a phone.
As a more general observation, tkinter GUIs in Pydroid 3 can take a bit of getting used to, and some are substantially easier to operate with a Galaxy Note-type stylus or Bluetooth mouse. For instance:
On the other hand, tkinter GUIs are surprisingly usable on a phone with practice—and might even be compelling.
Is that AndroWish under the hood?:
Programmers may be interested to know that the Pydroid 3
app's tkinter support might be based on the work of
project that uses the
graphics library to render Tk GUIs on Android devices, employs a
bridge between Android's Java code and Tk's C
code, and was itself based on an earlier project called SDLTk
(whose web whereabouts are unknown).
If so, this means that Pydroid 3's tkinter simply inherits both its utility
and flaws from AndroWish, though the Pydroid 3 commercial product also shares
responsibility for the software it bundles and sells.
Unfortunately, Pydroid 3's developer did not reply to multiple requests for verification of AndroWish use. To be sure, Pydroid 3's tkinter bears multiple signs of AndroWish-ness. Among them: its unique rightclick bindings smack of AndroWish's
sdltk touchtranslate command;
its GUIs-cannot-spawn-GUIs constraint discussed ahead exists in AndroWish too;
its tendency to fire multiple events for one gesture parallels an AndroWish bug report;
its install tree includes SDL components used by AndroWish;
and its overall look-and-feel seems the same as AndroWish's.
Still, this isn't enough evidence to constitute positive identification.
What can be said is that Pydroid 3's tkinter is likely based on work of others that is not necessarily tied to Pydroid 3, and might be usable in other Python coding apps, in standalone apps of your own, and even on other platforms. While intriguing, such prospects are both beyond this guide's scope, and largely moot. In truth, Pydroid 3 today brings Tk and tkinter on Android to the broad Python audience—something that the Tcl-biased AndroWish project seems unlikely to have ever pursued. Hence, we'll attribute Pydroid 3's tkinter to Pydroid 3 here because it's no less a part of this app than any other component it embeds, and note that running tkinter GUIs like Mergeall's on Android is a stellar achievement, regardless of its implementation.
The prior section paints a justifiably positive picture, but tkinter GUIs do not come without both procedural requirements and noticeable drawbacks in Pydroid 3. To bring Mergeall's GUI to life on your phone, you must first be willing to accept some constraints unique to Pydroid 3, and others common to Android in general. Specifically, you must:
Compared to the Termux approach, you do not need to install Python separately (it's part of Pydroid 3); install helper scripts (you'll use the GUI instead); or run a permissions setup command (Pydroid 3 requests permissions automatically).
However, you still need Mergeall's source-code package (you'll run its GUI's script); must patch two of its code files for Pydroid 3 alone (per the next section); and still have to work with Android version, filesystem, and permission constraints (per their earlier coverage).
While some of the steps above are self-explanatory, and we don't have space for an exhaustive dissection of the rest, the remainder of this appendix explores the most salient points on the list above.
Among the requirements just listed, the third may merit explanation most.
To use Mergeall's GUI in Pydroid 3, you must first download
Mergeall's source-code package from
this page to any
writeable folder on your phone, and unzip (i.e., extract) it there.
/sdcard/Download is the usual target for
downloads, and will suffice for both a quick look and later syncs.
There's more on Mergeall's source-code package
here and here
if you've jumped into this appendix randomly.
After installing the Mergeall source-code package, you'll need to replace two of its files, because it's not possible to integrate Pydroid 3's changes into the source-code package today for reasons explained ahead. To do this, simply click the following links to fetch custom versions of two Mergeall source-code files, and move or copy them into the top level of your unzipped Mergeall source-code package folder on your phone. That is, replace the originals with the versions below:
If you click these links online, you'll receive reply pages that looks like this; download to your phone by clicking the "save" in these pages' "Raw text" lines. If you're working offline, try a rightclick/longpress save, or visit the folder these links reference.
The changes applied in these files are minor, and apply to Mergeall's GUI only. Hence, they won't break Mergeall's command-lines mode described earlier in this guide if it's run from the same modified source-code folder. Conversely, these changes will break Mergeall on PCs, so use the files above only on your phone's copy of the source-code package. All changes made are marked with "# ANDROID" comments; search for this string in the custom files if you want to see what differs on Android.
In short, these files' changes address some but not all limitations of tkinter GUIs in Pydroid 3. For example, text labels were trimmed and control fonts shrunk for better fit on phones; some size settings were lowered for non-maximized mode; and a workaround was installed for a Pydroid 3 issue that prevented the GUI from spawning merges. Despite the changes, though, you must still avoid using some fonts in customizations because they trigger crashes; cannot open help or logfiles in the GUI itself due to a Python Android issue; and should not rotate some devices while Mergeall is running because this can hang Pydroid 3 altogether.
We'll take a closer look at these and other glitches in Pydroid 3's tkinter GUIs in a moment. First, though, let's run the GUI.
Once you've made the prior section's code changes, start Mergeall's GUI by opening
its source-code package's file
launch-mergeall-GUI.pyw in the
Pydroid 3 app's editor, and pressing the editor's big yellow run
Mergeall's GUI should appear, and work the same as it does on PCs—select FROM and TO folders, choose a REPORT or UPDATE, pick a logfile folder and other run options, and press the GUI's GO button to sync. Mergeall messages scroll in the GUI, and are saved to a logfile if you've requested one. As usual, you can open Mergeall's source-code file in either Pydroid 3 itself (see its folder icon), or most Android file explorers (tap the file and open with Pydroid 3).
As noted earlier, Mergeall tends to work best in fullscreen and maximized modes.
If desired, select these modes before running the GUI with the Settings ⇨ System
and press your phone's back button twice when you wish to exit Mergeall in
While you're in Pydroid 3's Settings, also consider using the Editor's monospace
font without line wrapping; Mergeall's code is otherwise difficult to view.
Viewing code is not required for the GUI, but you may wish to experiment with
configuration-file settings in
and the line between user and programmer is often gray at best.
This GUI approach works, but only for Mergeall syncs: other Mergeall
diffall.py comparisons and
cpall.py bulk copies
have no GUI, and are still best run with command lines in the Termux shell as covered
Pydroid 3 does have a Terminal option that is capable of running command lines too
(and can even run the main Mergeall script with a command typed
but it's much more limited than Termux for command-line work.
And while you can open and run such scripts in Pydroid 3's IDE, they won't work because
the IDE has no way to specify the command-line arguments they require.
As a special case, running Mergeall's GUI to sync to an empty folder has the
same effect as
cpall.py (and in fact uses the same code), but you won't
get as many status messages, and there's no GUI equivalent to
today. Command-line scripts need command lines (or an IDE that supports
Run GUIs faster with shortcuts:
Opening and running a source-code file in Pydroid 3's IDE normally requires multiple
steps—you must either open the app and navigate to your file, or navigate to
your file in a file explorer and open with the app. This, plus clicking the IDE's
run button, makes for substantially more steps than either a command in Termux or a
standalone app (and is one reason this guide downplays the GUI option).
That said, some Android file-explorer apps allow you to place a shortcut (a.k.a. link) to a source-code file on your home screen, like this and this. Once placed, starting a tkinter GUI program like Mergeall in Pydroid 3 is just two taps—click the shortcut, and click the IDE's run button. Pydroid 3 still has extra setup steps and glitches, but running a tkinter GUI can be nearly as easy as running an app. Usage notes: file explorers here and here are among those that make links, and be sure to set your file's default app or command to be Pydroid 3 in your file explorer (unless you like surprises more than monkeys).
Now that you know how to run a GUI sync on Android, you should also be aware up front that Pydroid 3's tkinter support may still not quite be ready for prime time. Running tkinter GUIs on Android is undoubtedly a major accomplishment, but the current implementation has some rough edges that Mergeall code changes cannot entirely smooth. While some don't impact Mergeall, and many are probably of more interest to programmers than to end users, the full set of issues confronting Pydroid 3 tkinter GUIs merits a summary here.
The exact scope of this issue is unknown. On some recent Samsung devices, for example, rotations in Pydroid 3 tkinter GUIs work fine if the display's screen-zoom setting is set to medium (not small), though this may hold true only after applying system updates, and screen zoom doesn't help on older phones. It may also be possible to work around this in Mergeall with a Java-based approach that locks orientation or ignores rotation, but this limits usage, and the underlying bug should really be fixed more globally for all programs. Ultimately, only Pydroid 3 tkinter GUIs exhibit this defect.
For now: if rotates don't work in tkinter GUIs on your device, either make sure your screen-zoom is not small and your device is up to date; wait for a fix in whatever software plays Brutus to Pydroid 3's rotation Caesar; or use Mergeall's GUI in either portrait or landscape orientation—only—and consider a rotation-locker app like one of these if you can't avoid flipping your phone.
webbrowsermodule for opening web pages and other content does not yet support Android. In Mergeall's GUI, this impacts both the "Help" and "Show logfile popup?" buttons, which must be disabled on Android alone. There may be a complex workaround alternative here too, but this bug's broad impact makes it better fixed in Python itself, and getting the GUI to work at all has taken priority thus far. For now, open Mergeall's user guide and logfiles outside the GUI.
helvetica); but most other font families including
systemtrigger an immediate and silent hardcrash, and italic and bold styles are simply ignored if submitted to tkinter in a font tuple instead of a string. Mergeall avoids crashes with font presets tailored for Pydroid 3, but customize with care (and see this file for names to use with platform-specific settings like fonts).
sys.executableis left empty by Pydroid 3, which in turn makes the
subprocess.Popenprocess-spawn code in Mergeall's GUI fail. Mergeall works around this by hardcoding this setting to use the path to Pydroid 3's Python, but this should be fixed in Pydroid 3: this same bug crippled most of the larger programs tested, though some other programs' spawns were also hobbled by the next issue. The workaround's change also applies to the Pydroid 3 app only, which makes it impossible to integrate into base files (more on this ahead).
This is fatal to programs like PyMailGUI that cannot be used without their launchers and cannot be easily recoded to use top-level windows instead of freestanding processes. Though its exact cause remains unknown, this restriction may be inherent in Pydroid 3 if its tkinter structure requires programs to be run by its IDE. Tellingly, tkinter programs also fail if run from Pydroid 3's own Terminal interface.
fromstatement on a line all by itself, or the script's GUI won't work (the required scaffolding appears to be unloaded). This didn't require a change in Mergeall, but the PyToe program in PyGadgets did not open in Pydroid 3 without one.
The PyEdit text editor, for example, has an explicit Quit button that works normally, but has no way to offer users a chance to save changes on double-back exits. Worse, the Frigcal calendar has no explicit quit widget of its own, and hence no way to save calendars in maximized mode. Non-maximized mode does better with a window-border exit button that runs exit handlers properly, but doesn't use available space as well, and can still be the scene of a silent-but-deadly double-back exit. The lesson here: don't use a double-back in Pydroid 3 unless doing nothing is what you mean to do.
multiprocessingparallel-execution module does not work on Android, because required semaphores are not supported on this platform. The impact of this will naturally vary program. PyEdit's Grep command, for instance, depends on this module on PCs; it can be easily configured to use fully supported threads on Android instead, but this makes it impossible to leverage multiple CPU cores (a speed factor that may be more important on PCs than phones). Other programs may require more extensive recoding.
<Return>event. This both precludes return-key actions in widgets and windows, and renders multiline-text widgets virtually unusable. It may reflect a bug in newer versions of the Samsung keyboard only (its older versions' return keys work fine), but Pydroid 3 seems to leave it in a state that fails for tkinter GUIs exclusively.
Luckily, there are at least three ways to work around this glitch if it impacts your device: use a physical Bluetooth keyboard; use Samsung's handwriting input option whose return key works correctly; or simply install an alternative on-screen keyboard whose return key works properly, such as Google's Gboard or Hacker's Keyboard. Both alternative keyboards support multiline-text input in programs like PyEdit and Frigcal, and the latter also helps with the next glitch on this list.
To work around this, use a Bluetooth keyboard, or install an alternative on-screen keyboard on your phone that can be opened on demand. Hacker's Keyboard, for instance, includes a permanent-notification option which allows you to pop up its on-screen keyboard at any time by clicking the app's entry in Android's notifications pulldown.
Of course, too much of a good thing can be a bad thing too: Android's automatic on-screen keyboards may also be an unwelcome feature in some tkinter GUIs. The Frigcal program, for example, required changes in its event handling to mute an annoying keyboard popup on most touch gestures. Any text field triggers the popup mindlessly—until it's silenced with uneditable state.
Ctrl+cto copy and
Ctrl+vto paste in keyboards that support these. You'll find support for these keys in both Bluetooth keyboards, as well as alternative on-screen keyboards like Hacker's Keyboard (yes, a pattern is emerging). This works, but may not be what a typical Android user would expect.
Altkey for menu shortcuts, in keyboards that support it
Probably the best spin on these is symptoms of still-maturing software.
There's more on the other programs noted in this list in the next section. The point here is that while some accommodation is to be expected when translating between PC and smartphone paradigms, issues like those above mean that running tkinter programs in Pydroid 3 is hardly seamless. Even allowing for fixes in its future releases, you'll clearly want to test-drive this app on tkinter programs you'd like to use to see how its support applies.
Despite the workarounds and glitches of the preceding section, the second to last of the Pydroid 3 requirements listed earlier is really its most regrettable and grievous downside: the free version of Pydroid 3 will flash distracting and distasteful fullscreen ads in your face every few runs, unless and until you pay a required $9.99 freemium fee. To be blunt, this comes off as ransom, and borders on extortion. It may be a norm in Android's culture, but it is rude and unacceptable in Python's, and makes it difficult to recommend Pydroid 3 to Mergeall users today.
Together with the prior section's glitches, this also makes it impossible to
integrate the changes required for Pydroid 3 tkinter GUIs into source-code
package base files. For example, the coding workarounds that accommodate
Pydroid 3's font crashes are specific to Pydroid 3 alone,
sys.executable hardcoding will work for no other app.
Changing portable source-code packages
to support just a single app on Android is ethically dubious at
best—and just plain wrong when that app employs advertising.
Though less convenient, Pydroid 3 today is best supported
with the custom source-code files used here.
This guide's broader advice to Mergeall's users is to evaluate its GUI in Pydroid 3 to see if it warrants the fee in your context. It may, especially if you have other tkinter programs you wish to run on Android. As examples of this app's broader scope:
Wherever you may land on this issue, there's no denying that Pydroid 3's advertising scheme casts programs like Mergeall in a supporting sales role that's both uncomfortable and unseemly. A free trial period would be much more palatable than obtrusive advertising, and will hopefully be a part of the future of the otherwise-remarkable Pydroid 3 app. Barring that, let's hope that free and open-source alternatives for running the free and open-source tkinter appear for Android soon.
It's a large world after all: Naturally, there may be other tkinter-on-Android options afoot or unknown at this guide's publication, and others may appear in the future. Please exercise the usual due diligence, and check back here for updates as change warrants and time allows. This guide wishes to be inclusive, but the quality of web-search results has grown inversely proportional to the greed of companies providing them.
Finally, in addition to all the Pydroid 3 requirements described in the preceding sections, you must also keep in mind that this appendix's GUI approach shares all the other general constraints that this guide notes for the Termux command-lines approach. Specifically, to use Mergeall's GUI in Pydroid 3, you'll still need to:
In other words, Mergeall's GUI may run on Android, but it's still running on Android.
More on Pydroid 3 storage permissions:
Notice the last half of the general requirements listed in this section—it arises because
Pydroid 3 shares the exact same permission constraints that we saw earlier
for Termux. Namely, the programs that Pydroid 3 runs can read anywhere on internal storage
and removable media, and can write anywhere on internal storage, but writes on
removable media are limited to Pydroid 3's app-specific folder only, which is at the
following path on your
/storage/xxxx-xxxx drive (once you create it,
and swap the
xs for your drive's Android ID):
It's easy to verify this: see the permissions test script, and its results on both Pydroid 3 and Termux; the results do not differ between Android Nougat and Oreo and are unchanged by installing Pydroid 3's extra-permissions plugin. The takeaway from these results is that storage access rules are the same for both apps, and updates on removable media are limited in both to the app's own folder. The only real difference in the storage-permissions department is that Pydroid 3 replaces Termux's one-time script run with a one-time automatic request on first access to restricted folders.
As we've learned, this means that content on removable media must be nested in Pydroid 3's app-specific folder if it is to be updated by Mergeall in this app. It also means that items to be updated by both Termux and Pydroid 3 must be located in internal storage, because these apps can modify only their own app-specific folders on removable media. Importantly, your choice of Mergeall content folder on SD cards and USB drives wholly determines which access-limited app can change it. For instance, if Mergeall's GUI in Pydroid 3 processes removable-drive content in Termux folders—as in earlier screenshots—it is limited to comparisons only. And while our focus here is on Mergeall, this constraint applies to any GUI run in the app.
This is an unavoidable downside to using Android removable storage in these apps. The lesson here is that internal storage—usually at
always be used if possible, because it can be changed by both Termux and Pydroid 3
specifically, and by other apps generally. As an example of the latter set,
a Python Android app not covered in this guide because
it holds no advantages for Mergeall today, has the same storage-access limits per
Android's rules must be hard to beat.
So there it is: the good, the bad, and the ugly of using tkinter GUIs like Mergeall's in the Pydroid 3 Android app. To be sure, the tkinter support embedded in this app is nothing short of stunning: it implements nearly all of Tk and tkinter on smartphones, and makes these devices substantially more fun to use, for both programmers and others. You'll have to decide for yourself whether its remaining defects—along with the advertising model of the app that brings it to you—justifies using this support on your phone.
Beyond Pydroid 3's tkinter support, there are additional GUI options for Python programs on Android that we don't have space to cover here. For example, while Termux does not offer tkinter support directly, it does provide some X Windows (a.k.a. X11) GUI support that may provide a path to using tkinter in its Linux environment. For a survey, see its links here and here, and the X11 search here. Even if viable, however, X11 support requires substantial extra setup work that may be too much to ask of users running Mergeall alone.
Other GUI options offer Android-friendlier alternatives, including Kivy, SL4A, and Qt's PyQt and PySide. Such tools allow programs to use Python for their core logic, but wrap it in GUIs that more closely match the Android paradigm. Although their program recoding costs are not trivial, these tools become more attractive for tkinter GUIs that must be redesigned for smaller screens if their audiences are ever to include more than the most stubborn tkinter fans.
At the end of the day, Pydroid 3's tkinter support is admirable and even amazing, but the approach it offers for Mergeall really just swaps command lines for a GUI run in an IDE that requires extra code changes and comes with noticeable seams. Whether you find it easier to sync your content by running a single command in Termux, or opening and running a source-code file in the Pydroid 3 IDE may very well depend on your background. For this guide's money, though, the latter is just enough extra steps to qualify as runner-up.