The availability of vast amounts of operational and end-user data in cyber-physical systems implies that paradigm improvements in monitoring and control can be attained via learning by many artificial intelligence agents despite them possessing vastly different abilities. Engaging this heterogeneous agent base in the context of the smart grid requires the use of hierarchical markets, wherein end-users participate in downstream markets collectively through aggregators, who in turn are coordinated by an upstream market. The goal of this project is to conduct a systematic study of such market-mediated learning and control. This project aims at much deeper levels of participation from end-users contributing electricity generation such as rooftop solar, shedding load via demand response, and providing storage capabilities such as electric vehicle batteries, to transform into reliable distributed energy resources (DER) at the level of wholesale markets. A methodological theme is multi-agent reinforcement learning (MARL) by agents that control physical systems via actions at different levels of the hierarchy. Underlying the whole project are well-founded physical models of the transmission and distribution grids, which provide structure to the problem domain and concrete use cases. This project facilitates a deeper level of decarbonization in the electricity sector, and contributes to climate change solutions by engineering a flat, interactive grid architecture that allows significant DERs to provide electricity services to both local and regional grids. Engagement with a grid-level market operator enables the project to address a problem space of immediate relevance to the current electricity grid. The project also includes the development of educational materials on data-analytics and energy systems. Intrinsic to the program are efforts at outreach to involve high-school students via demonstrations and lectures based on the technology developed.