Yes, most tarballs do not support random access (there are some metadata extensions that allow this). This makes large tarballs annoying to use on systems with slow disk I/O (even a hard disk may be too slow (to the degree of being annoying to work with)). This is by far my biggest gripe with the format. Certainly, smaller tarballs are a very handy format as long as you stay inside the Unixy world of computing – and as long as you keep looking out for the various incompatibilities between the different tar implementations.
"... there are some metadata extensions that allow this)."
Where to find these extensions? Are they portable between Linux and BSD?
The 1998 dict project included a utility called "dictzip" for random access to the contents of gzip compressed files.
Dumb question: Is it possible to create a utility or even a hack that performs "random access" into tar archives?
Example use case: the user only wants to untar a small number of selected files from a large tarball such as a source tree.
The user has tried both the "-T filelist" option and using memory file systems instead of hard disk drives.
A zip file is a concatenation of gzipped files. A .tar.gz is a gzip stream of concatenated files. Anything that could do random access into the contents of a zip file entry could do similar things with a tarball.
With a transparent random access overlay, the difference mostly disappears, reducing to whether the stream needs to be scanned or whether it's indexed, which is itself orthogonal - zip file directory at the end is redundant.
So you mean at each "random access", you actually have to scan the whole .tar.gz file to find the location? For large tarballs, that will definitely hinder performance a lot. The difference does not disappear at all.
AFAIK a compressor like zip builds a dynamic running table of frequent byte sequences; the resulting archive is written in such a way that when you decompress it, you re-build the table in the process.
So if you concatenate files A, B, and C and then compress the result, then by the time the compressor starts compressing the data of C, it will have that table built from A and B. To extract C, you'll need to re-build the same table and thus you'll need first to decompress A and B.
In a zip file each entry is compressed individually; this gives random access, but worse compression rate, because the table is not re-used between files.
Tape drives don't really support random access, though, which is reflected in the design of the tar format and its offspring. That is, in fact, the problem here, and why formats designed for random access instead of sequential access are far better for storing file systems for containers and VMs.
