Use inference manager for nl_solver (#5125)