The Fold3 and the Deltas Fix

Published October, 2021

This page is a continuation of the Android 11 saga started here and left here. It both reviews the prospects for keeping content on a new Samsung flagship phone, and details a work-around that was developed to manage content with Mergeall on this device.

The work-around chronicled here is a manual process which requires a new script and a dedicated proxy drive, and is nowhere near as convenient as either USB access or microSD cards available earlier. But it passes as a fix of sorts, and can be used with any Android 11 and later device that has been similarly handicapped by both its hardware and software makers (at this writing, Android 12 is still maimed).

A note up-front: this page does not reflect a vested interest of any kind in Samsung or its products. The phone focus here stems from a first-hand evaluation, and is rooted in the fact that this company has been a leading Android vendor for years. While other phones' mileage may vary, the reality is that Samsung's moves have almost as much weight on the Android world as those of Android itself.

Samsung Cripples Its Flagships by Removing microSD

In August 2021, Samsung released its newest flagship Android phone, the Galaxy Z Fold3 5G. This device is clearly interesting from a hardware perspective—it's both phone and mini tablet, with an amazing screen that brings content creation into scope, and makes it difficult to go back to the standard candy-bar form factor and its porthole views.

In a word, this phone is disruptive. Its cameras have room for improvement, and time will tell if smartphone users accustomed to placing their phones in situations as perilous as a back pocket will warm to the new folding paradigm. But this device already feels like the future.

The Bad News

At least at first glance, however, the Fold3 also seems to be unusable as a host for non-trivial content collections managed by portable programs like the Mergeall content-sync system. Despite its new-age aesthetics, this device:

The combo of these two yields a content-island device, with neither removable storage nor open USB access. You can't pop a card out and manage it on your PC, and you can't plug in a USB drive and access it from POSIX-based programs run on the phone. Hence, there is no direct way to store and manage large collections of content on this device using Python programs like Mergeall. This is true for the large 200G collection maintained by Mergeall's proprietor, but much smaller collections will fare the same.

The Damage

To be clear, USB drive access isn't fully gone in Android 11: it's just been locked down so it can be accessed only through Android's proprietary Java framework. Programs coded in Java, such as file explorers, can still do so, though many or most undoubtedly required significant changes to work in 11 (and some still do). This, however, qualifies as a kiss of death for portable POSIX-based programs like those coded in Python, which rely on device-neutral pathnames and system calls to work across multiple platforms.

Tools and techniques that allow Python code to call out to Java code have long been available, and might in principle make it possible to interface with Android's Java frameworks to access USB content. However, while findings are still tentative, this seems likely to be:

Hence, as of Android 11, there is no direct way to access USB-drive content from Python code. And there's no clear way to use any POSIX-based software to update on-phone content through the USB port. Samsung's removal of microSD support appears to seal content's last escape hatch, and finish the work that Android 11 began.

The Options

It's difficult not to see this as a vendor closing the doors that provided alternatives to cloud storage. With either removable microSD or unlocked USB, content could be stored and maintained on Android 11 phones, in either shared storage or app-specific storage. To be sure, both options have substantial tradeoffs in Android 11:

Despite their downsides, both options are at least viable, and reflect unfortunate but workable sacrifices for on-phone content storage in Android 11, made arguably more palatable by devices like the Fold3.

But with neither microSD nor USB, internal on-phone storage seems a moot point. There's no direct way for a Python script to get the data onto a phone, and no way for a Python script to incrementally update the data once there. As we saw here, network-based solutions are far too slow or buggy to be practical for non-trivial content; and file-explorer copies by USB, when they work at all, may botch modification times, permissions, and more, and won't help for later content-change synchronizations in any event.

Conclusions 1.0

Thus, the Fold3, like most of Samsung's other recent releases, initially appears to be unusable for storing and maintaining large content collections with POSIX software. It may be viable for smaller content sets for which full drag-and-drop copies suffice to propagate changes, and for which cloud options are feasible—and this, alas, may be one of the real goals of this device. But larger collections, at least on first inspection, are simply out of scope on this phone.

More fundamentally, this phone ships with 256G or 512G internal storage, and no good means to populate it, apart from occasional media downloads, exhaustive and perilous brute-force copies, and software that slavishly complies with Android's increasingly draconian requirements. That's a kind of device, but it's not a PC replacement. While some other vendors still ship phones with microSD support, the fact that the leading Android hardware maker is now crafting content islands seems a likely and regrettable harbinger of Android's closed future.

A Mergeall Work-Around to the Rescue: Delta Sets

All the forgoing being said, the sky hasn't fallen in full.

For users of the Fold3—and all other content islands like it—the Mergeall team has come up with a new way to use Mergeall to update on-phone content to match that on a PC (where PC means any Windows, macOS, or Linux device). It's a manual procedure that uses custom code and a new script that batches changes, and requires a dedicated external drive to serve as a proxy for the phone. But it does not require POSIX USB access or microSD card, and does not rely on phone rooting or any of the impractically slow network-based alternatives to USB which we explored earlier.

Work-Around Overview

This fix is largely automated by scripts we'll meet in moment, but as a summary, here are its core concepts and steps:

Setup devices
This fix requires Python, Mergeall's source code, and the scripts' package on PC and phone, along with some simple edits to config files to specify content paths.
Reserve a proxy
This fix also requires a USB drive to hold a copy of your content which will serve as a stand-in for the phone. This drive is called a "proxy" because it takes your phone's place when syncing from your PC.
Copy content
To get started, you'll run a script on your PC and phone to initially copy your content to a proxy drive, and from there to your phone. This requires a manual copy step on the phone because POSIX scripts can't access USB, but you'll copy just one content zipfile.
Sync changes
For all later syncs, you'll run a script on your PC and phone to merge all the changes made on your PC into the content copies on both the proxy and your phone. Again, this requires a manual copy step on the phone, but for just a single changes zipfile.

Beyond this, additional scripts are provided for verifying and exporting content stored on your phone, though these are optional and occasional tasks.

The Deltas Script

Much of the "magic" behind this work-around is the new Mergeall script, deltas.py. This is a variation of the main mergeall.py script. It collects changes (a.k.a. deltas) required to make the TO folder the same as FROM as usual, but has been modified to store these changes in a folder as a separate set, instead of applying them to TO immediately. The deltas script was initially coded to collect and archive recent changes to data already burned to optical disk, but it can be an advantage anytime a deferred changes set is useful.

In particular, delta sets have two crucial features for our purposes here. First, because they contain only recent changes, delta sets are generally very small, and can be copied much quicker than full content. Second, because deltas are stored in the same format as Mergeall backups, the main mergeall.py script's -restore mode can be used to apply them on demand to any content copy that's the same as the copy used to compute the deltas.

This combination of features makes deltas ideal for USB-handicapped phones. Specifically, deltas can be:

The net effect brings the phone up to date with the PC's changes indirectly, and requires neither POSIX USB access nor microSD card support. In addition, zipping deltas can make them immune to platform and filesystem interoperability issues in transit, and spare them from the vagaries (and outright failures) of raw copies in Android file-explorer apps.

The Work-Around

The best news is that nearly all of the above is automated by free and open-source Python scripts, which you can run with either a simple command line, a file-icon or shortcut click, or an easy step in an IDE GUI. Rather than going into further details here, though, this page will defer to these scripts' own documentation and content for more coverage. Get the fix and its usage docs here:

Usage guide

Package content

Package zipfile

In closing, it's important to note that this workaround is probably not for casual phone users. Though automated, it requires running command lines or script files, and is nowhere as nice as the former two-merge solution that can be launched with a GUI on both sides of the fence and is still usable on nearly every other platform—including Windows, macOS, Linux, and Androids 10 and earlier.

Android 11 remains the odd platform out in this story (along with the iOS content prison it seems determined to emulate). While 11's restrictions make content-management tasks harder, this fix's scripts at least make them feasible for those who understand that the perils of cloud storage easily justify extra measures to ensure the privacy of your digital property. This is, after all, Mergeall's main point.

 

[Python Logo] Top Deltas Android11 Android Tkinter Mergeall Mergeall Input ©M.Lutz