A theory is the more impressive the greater the simplicity of its premises, the more different kinds of things it relates, and the more extended its area of applicability.
In researching the origin of the quote, I discovered it is paraphrased in an alternative way:
[A law] is more impressive the greater the simplicity of its premises, the more different are the kinds of things it relates, and the more extended its range of applicability. [differences emphasized]
In either case, the description of the power (or impressiveness) of a theory (or law) exactly matches the description of a spanning layer, a phrase coined by David D. Clark, former chair of the Internet Architecture Board (IAB):
Certain protocols are designed with the specific purpose of bridging differences at the lower layers, so that common agreements are not required there. Instead, the layer provides the definitions that permit translation to occur between a range of services or technologies used below. Thus, in somewhat abstract terms, at and above such a layer common standards contribute to interoperation, while below the layer translation is used. Such a layer is called a “spanning layer” in this paper. Interoperation, Open Interfaces, and Protocol Architecture
Huh? This doesn't sound anything like Einstein's quote. Well, what
The [spanning layer] is shown as the narrow point in the hourglass to illustrate that there must be a narrowing of the range of alternatives at that layer. There can be a wide range of applications supported over the [spanning layer] and of technologies utilized under it, but the [spanning layer] itself must represent the point at which there is a single definition of the provided capabilities. The power of the scheme is that programmers for all the applications write code that depends only on this single set of capabilities and thus are indifferent to the actual technology used below. If instead there were competing alternatives in the definition of the [spanning layer], the application programmers would have to cope with this variation, which in the limit is no more useful than having applications deal directly with the individual network technologies. In computer science terms, if there are N applications and M technologies, the [spanning layer] reduces the complexity of the resulting situation from N M to N + M. But for this to succeed, the [spanning layer] must be a point of agreement on a single set of capabilities.
I hope that the underlined section makes clear the connection between Einstein's description of an impressive theory and
A simple set of loosely coupled standard formats, protocols, and identifiers that generalizes diverse technologies into a virtual service for use by diverse applications
As you can see, at both the conceptual and technological level, generalization and unification are core concepts. This is a deep aspect of loose coupling in science as well as technology. To understand how spanning layers evolve is to understand evolution more deeply.
One other aspect of Einstein's quote is the context in which he made it (or at least one of the contexts—he seems to have used the quote several times). Here is a fuller quote:
A theory is more impressive the greater the simplicity of its premises, the more different are the kinds of things it relates, and the more extended its range of applicability. Therefore, the deep impression which classical thermodynamics made on me. It is the only physical theory of universal content, which I am convinced, that within the framework of applicability of its basic concepts will never be overthrown. M.J. Klein, Thermodynamics in Einstein's Universe, in Science, 157 (1967), p. 509
Here is a slightly different version of the quote:
A theory is the more impressive the greater the simplicity of its premises, the more different kinds of things it relates, and the more extended its area of applicability. Therefore the deep impression that classical thermodynamics made upon me. It is the only physical theory of universal content which I am convinced will never be overthrown, within the framework of applicability of its basic concepts. [differences emphasized]
Given my belief that entropy drives the evolution of complexity via spanning layers, such a quote gives me hope that I am on the right track.