We present the complete four-loop four-point amplitude in N=4 super-Yang-Mills theory, for a general gauge group and general D-dimensional covariant kinematics, and including all non-planar contributions. We use the method of maximal cuts --- an efficient application of the unitarity method --- to construct the result in terms of 50 four-loop integrals. We give graphical rules, valid in D-dimensions, for obtaining various non-planar contributions from previously-determined terms. We examine the ultraviolet behavior of the amplitude near D=11/2. The non-planar terms are as well-behaved in the ultraviolet as the planar terms. However, in the color decomposition of the three- and four-loop amplitude for an SU(N_c) gauge group, the coefficients of the double-trace terms are better behaved in the ultraviolet than are the single-trace terms. The results from this paper were an important step toward obtaining the corresponding amplitude in N=8 supergravity, which confirmed the existence of cancellations beyond those needed for ultraviolet finiteness at four loops in four dimensions. Evaluation of the loop integrals near D=4 would permit tests of recent conjectures and results concerning the infrared behavior of four-dimensional massless gauge theory. Show Partial Abstract