As suggested by recent experimental evidence, a spontaneously active neural system that is capable of continual learning should also be capable of homeostasis of both activity and connectivity. The connectivity appears to be maintained at a level that is optimal for information transmission and storage. We present a simple stochastic computational Hebbian learning model that incorporates homeostasis of both activity and connectivity, and we explore its stability and connectivity properties. We find that homeostasis of activity and connectivity imposes structural and dynamic constraints on the behavior of the system. For instance, the connectivity pattern is sparse and activation patterns are scale-free. Additionally, homeostasis of connectivity must occur on a timescale faster than homeostasis of activity. We demonstrate the clinical relevance of these constraints by simulating a prolonged seizure and acute deafferentation. Based on our simulations, we predict that in both the post-seizure and post-deafferentation states, the system is over-connected and, hence, epileptogenic. We further predict that interventions that boost spontaneous activity should be protective against epileptogenesis, while interventions that boost stimulated or connectivity-related activity are pro-epileptogenic.