Natural stimuli often have time varying first (i.e. mean) and second order (i.e. variance) attributes that each carry critical information for perception and can vary independently over orders of magnitude. We recorded the responses of midbrain electrosensory neurons in the weakly electric fish Apteronotus leptorhynchus to stimuli with first and second order attributes that varied independently in time. We found two distinct groups of midbrain neurons: the first group responded to both first and second order attributes while the other responded selectively to second order attributes. Using computational analyses, we show how inputs from a heterogeneous population of ON- and OFF-type afferent neurons are combined in order to give rise to response selectivity to second order stimulus attributes in midbrain neurons. Our study thus uncovers, for the first time, generic and widely applicable mechanisms by which selectivity to second order stimulus attributes emerges in the brain.