The large-scale structure (LSS) of the universe provides a direct window into the nature of dark matter, dark energy, and the physics of inflation. Through current and future galaxy survey observations, LSS will provide the lion's share of new information used to constrain fundamental physics in cosmology. Extracting physical information from this data is complicated by non-linear gravitational evolution and galaxy formation. These complex processes must be modeled using expensive numerical simulations, which account for some of the largest jobs run on DoE high-performance computing facilities. Such necessary simulations only grow in expense with time due to higher-resolution imaging and larger survey volumes. These challenges of scale will be especially relevant for the analysis of data from DoE missions such as Rubin Observatory, the Dark Energy Spectroscopic Instrument, and CMB-S4. The extreme computational requirements of accurate models of LSS demand strategies to make statistical inference of physical parameters tractable. I will present multiple approaches to making the LSS inference problem soluble. Differentiable forward models can make use of efficient gradient-based high dimensional optimization and sampling algorithms. While differentiable N-body solvers exist, such simulations require the solution of the Einstein-Boltzmann equations to generate their initial conditions. I will present the first differentiable solver for these ODEs that I developed with a collaborator. In the absence of gradients, an alternative strategy is to directly approximate the posterior over the model parameters using a cheap surrogate model to accelerate the inference procedure. With my advisor, I have developed a novel algorithm for performing surrogate optimization of expensive posterior objectives using Gaussian Processes and Normalizing Flows - I will present this model and its applications. Finally, I will present preliminary results from ongoing thesis work that uses a combination of these approaches to infer limits on the physics of inflation in the early universe.