Latent variable models have been widely applied for the analysis and visualization of time series that result from techniques in experimental neuroscience. These are datasets in which the evolution of the observations is relatively smooth and possibly nonlinear. In this work we present Variational Inference for Nonlinear Dynamics (VIND), a variational inference framework that is able to uncover nonlinear, smooth latent dynamics from sequential data. The framework is a direct extension of PfLDS; including a structured approximate posterior describing spatially-dependent linear dynamics, as well as an algorithm that re- lies on the fixed-point iteration method to achieve convergence. We apply VIND to electrophysiology, single-cell voltage and widefield imaging datasets with state-of-the-art results in reconstruction error. In single-cell voltage data, VIND finds a 5D latent space, with variables akin to those of Hodgkin-Huxley-like models. The quality of VIND’s learned dynamics is further quantified by using it to predict future neural activity. VIND excels in this task, in some cases substantially outperforming current methods.