The early chemical evolution of the Galaxy and the Universe is vital to our understanding of a host of astrophysical phenomena. Since the most metal-poor Galactic stars are relics from the high-redshift Universe, they probe the chemical and dynamical conditions as the Milky Way began to form, the origin and evolution of the elements, and the physics of nucleosynthesis. They also provide constraints on the nature of the first stars, their associated supernovae and initial mass function, and early star and galaxy formation. I will discuss examples of the most metal-poor Galactic stars with extreme and unusual abundance patterns that can help elucidate the supernovae responsible for their chemical signatures. Furthermore, stars displaying a strong overabundance of the heaviest elements, in particular uranium and thorium, can be radioactively dated, giving formation times ~13 Gyr ago, similar to the ~13.7 Gyr age of the Universe. I then transition to a description of recent discoveries of extremely metal-poor stars in dwarf satellites of the Milky Way. Their stellar chemical signatures support the concept that small systems analogous to the surviving dwarf galaxies were the building blocks of the Milky Way's low-metallicity halo. This opens a new window for studying galaxy formation through stellar chemistry.