Abstract
Background
According to the classical model of Macevicz and Oster, annual eusocial insects should show a clear dichotomous "bangbang" strategy of resource allocation; colony fitness is maximised when a period of pure colony growth (exclusive production of workers) is followed by a single reproductive period characterised by the exclusive production of sexuals. However, in several species graded investment strategies with a simultaneous production of workers and sexuals have been observed. Such deviations from the "bangbang" strategy are usually interpreted as an adaptive (bethedging) response to environmental fluctuations such as variation in season length or food availability.
To generate predictions about the optimal investment pattern of insect colonies in fluctuating environments, we slightly modified Macevicz and Oster's classical model of annual colony dynamics and used a dynamic programming approach nested into a recurrence procedure for the solution of the stochastic optimal control problem.
Results
1) The optimal switching time between pure colony growth and the exclusive production of sexuals decreases with increasing environmental variance. 2) Yet, for reasonable levels of environmental fluctuations no deviation from the typical bangbang strategy is predicted. 3) Model calculations for the halictid bee Lasioglossum malachurum reveal that bethedging is not likely to be the reason for the graded allocation into sexuals versus workers observed in this species. 4) When environmental variance reaches a critical level our model predicts an abrupt change from dichotomous behaviour to graded allocation strategies, but the transition between colony growth and production of sexuals is not necessarily monotonic. Both, the critical level of environmental variance as well as the characteristic pattern of resource allocation strongly depend on the type of function used to describe environmental fluctuations.
Conclusion
Up to now bethedging as an evolutionary response to variation in season length has been the main argument to explain field observations of graded resource allocation in annual eusocial insect species. However, our model shows that the effect of moderate fluctuations of environmental conditions does not select for deviation from the classical bangbang strategy and that the evolution of graded allocation strategies can be triggered only by extreme fluctuations. Detailed quantitative observations on resource allocation in eusocial insects are needed to analyse the relevance of alternative explanations, e.g. logistic colony growth or reproductive conflict between queen and workers, for the evolution of graded allocation strategies.
Background
The optimal allocation of accumulated resources to maintenance, growth, and reproduction is the central topic of life history theory. At any time during its life an organism must decide whether it will allocate available resources to maintenance, to somatic growth (that will allow for larger reproductive potential in the future), or to reproduction. In particular, the existence of a tradeoff between growth and reproduction has been well confirmed [1,2]. Much theoretical and field work has been invested to understand the pattern of investment into growth and reproduction and to predict which allocation strategies will maximise an organism's fitness [35]. Since the first paper by Cole [6] theoretical analysis of life history strategies has focused on solitary organisms (for reviews see [1,2,4,7]). In contrast, since the eminent work of Macevicz and Oster [8] the evolutionary analysis of nest cycle dynamics in social species has not gained much further attention [9,10]. For social insects the problem of an optimised investment into growth and reproduction mostly concerns the growth of the colony as a whole; how much resources should be allocated to increase worker number and how much to the production of sexuals? As the answer to this problem strongly depends on the time left until the end of the season and as this quantity continuously changes we refer to optimal investment patterns as dynamic strategies. Dynamic allocation strategies in eusocial insects have first been analysed by Macevicz and Oster [8] and Oster and Wilson [11].
Macevicz and Oster [8] analysed the prototype of an annual eusocial colony cycle as exhibited by many vespid wasps, bumble bees and halictid bees and calculated optimal resource allocation strategies for the case of predictable or constant season length [8,12]. When season length is fixed and conditions are constant during the season the predicted optimal investment pattern is a simple "bangbang" strategy with the annual productivity cycle divided into two phases; colonies should start with a phase of pure colony growth, i.e. the exclusive production of workers, and – at some time – abruptly switch to a purely reproductive phase with the exclusive production of male and female sexuals. The optimal moment to switch between the two phases is entirely determined by season length, worker productivity rate, and worker mortality rate.
However, already Greene [13] has pointed out that colony development of many annual eusocial insects does not conform to the predicted bangbang strategy but is characterised by a gradual shift from the production of workers to the production of sexuals. Such "graded control" has been reported in wasps [1319], bumble bees [20,21] and halictids [2225].
Although sufficiently detailed quantitative data are hardly available, the halictid bee Lasioglossum malachurum can serve as one example of this type of colony dynamics. Recent studies of L. malachurum around Wuerzburg (northern Bavaria, Germany) provide daybyday observations of the timing of reproduction in five colonies [24,26]. These data clearly demonstrate the existence of graded control in this species and allow quantifying the length of the transition phase, when there is a simultaneous production of workers and sexuals.
Graded resource allocation strategies like that observed in L. malachurum are often interpreted as an evolutionary, risk spreading response (bethedging) to environmental stochasticity [1,2]. If the complete population under consideration suffers from identical but unpredictable year to year variation in productivity, mortality, or season length we expect bethedging strategies to be favoured by natural selection [27,28]. For example, in response to fluctuating season length plants may produce offspring with different diapause strategies [27,29]. Crickets may produce micropterous as well as macropterous individuals in response to variable availability of annual thermal energy [30], and young mice and voles may vary in age of maturity within populations in dynamic environments [31]. For solitary insects Hopper [32] has reviewed numerous cases where the occurrence of mixed strategies has been linked to the spreading of risk.
Oster and Wilson (1978) were the first to apply the general argument of bethedging to colony dynamics. They suggested that it could be the ultimate mechanism responsible for the evolution of graded control in social insects: "It can be demonstrated that stochastic variation in the system parameters will always promote graded control [11]." Following Oster (Fig. 2.16 in [11]) a risk spreading investment strategy in eusocial insects would be realised as a gradual sigmoid (instead of a dichotomous) transition between worker and sexual production. As a correlation between variation in season length (as a specific and common example of environmental fluctuations) and graded strategies is supported by many studies [30,33,34], variation in season length is often invoked to explain the occurrence of graded control in annual eusocial insects [9,11,17,22,23].
Hopper [32] reviewed several (potential) strategies of riskspreading in solitary organisms: temporal, metapopulation, and withingeneration spreading of risk. He concludes that the empirical evidence in support of bethedging as an important driver for the evolution of facultative diapause, migration polyphenism, spatial distribution of oviposition, egg size, and other traits is weak or doubtful, and that interannual environmental variability often turns out to be too weak to favour (substantial) riskspreading. As the plausible verbal arguments of Oster and Wilson [11] in favour of bet hedging as the ultimate cause for graded control have never been worked out in detail, it remains an open question as to whether environmental fluctuations are sufficient to explain the evolution of graded investment strategies in eusocial insects.
In this paper we present a formal analysis of the influence of environmental stochasticity (implemented as fluctuations in season length) on the investment strategy of social insects. Based on the colony model of Macevicz and Oster [8] we investigate how the optimal strategy of temporal resource allocation is influenced by the distribution of environmental conditions (mean, variance and shape of the distribution of expected season length). For the special case of Lasioglossum malachurum we derive estimates of the variability of seasons. They allow us to predict optimal temporal resource allocation for L. malachurum and to check whether environmental fluctuations can explain the broad transition phase between colony growth and reproduction sometimes observed in this species.
Results
Deterministic environments
Our analysis is organised in two steps. First, we present a deterministic model of the colony cycle with constant season length. As numerical optimisation methods are required later and the nocturnal inactivity of the colonies provides a natural time base, we use a timediscrete version of the classical model of Macevicz and Oster [8] with a time step of one day. Secondly, we calculate the optimal investment strategy when season length varies according to a given distribution.
Our model represents colony development during a single season of length L (this condition will be relaxed later). Two main dependent variables describe the state of a colony: the number of workers (W_{i}) and the number of sexuals (S_{i}) at time step i. The colony cycle typically starts in spring with nest founding by inseminated and hibernated females. During the founding phase the females work alone and perform all those foraging tasks that will be taken over by workers after their emergence later in the season [12]. Thus, we start with initial condition W_{1 }= 1 assuming that the founding queen acts like a single worker until the first eggs have developed to adults [35]. The change in the number of nestmates is governed by two mechanisms: mortality and reproduction. Each individual survives from time step i to i + 1 with probability q. Resource allocation in each time step (i) is directly proportional to the current worker force (W_{i}). Each worker can provision c (worker efficiency) eggs (= brood cells) per time step. For the sake of simplicity survival and efficiency of individuals are assumed to be constant during the whole season. We further assume that the actual egg laying rate of the queen is not limited, but the number of eggs that can be successfully provisioned depends (linearly) on the number of workers in the colony [35]. A time dependent fraction (u_{i}) of resources is allocated to the production of sexuals while the fraction (1u_{i}) is invested into the production of workers. Thus, the number of workers (W_{i + 1}) at time step i + 1 can be calculated as
In most eusocial halictid bees, annual vespid wasps, and bumble bees the life span of queens is much longer than life span of workers [12,36]. Female sexuals have to hibernate before nest founding in the following year, while workers live only for several weeks. Thus, we neglect mortality of sexuals as has been done by Macevicz and Oster [8] in most of their analyses. Consequently the number of sexuals (S_{i + 1}) at time step i + 1 can be calculated as
Like Macevicz and Oster we do not differentiate between investment in male and female sexuals. Thus S_{i }is the total investment in both sexes and as the cost of male and female sexuals differs significantly in halictids [24] the total number of sexuals will depend on the sex ratio produced by the colony. As we do not want to complicate the paper by including sex ratio considerations we will in the following – according to Macevicz and Oster – simply call S the number of sexuals.
These two equations fully determine the development of an annual primitively eusocial bee colony from nest founding at time step i = 0 until the end of the season (i = L). The fitness of colonies following such nest dynamics can be measured by the final number of sexuals successfully raised (S_{L}). Macevicz and Oster [8] as well as Oster and Wilson [11] have studied such systems (in time continuous form) as optimal control problems with control variable u_{i }(fraction of resources allocated to the production of sexuals). In the deterministic case (when season length L does not change between years) they found that the (timedependent) optimal control solution (u_{i}) that maximises S_{L }is a dichotomous bangbang strategy and u_{i }should switch from 0 to 1 at an optimal switching time (SWT). Thus, the optimal temporal pattern of reproduction consists of two distinct phases, a growth phase with exclusive worker production (u_{i }= 0 for 1 ≤ i <SWT) followed by a reproductive phase characterised by the exclusive production of sexuals (u_{i }= 1 for SWT <i ≤ L). Accordingly, the optimal strategy can be characterised by a single parameter, the optimal switching time SWT ∈{1, 2, ..., L}, when sexual reproduction begins and u_{i }(i ∈ {1, 2, ..., L}) changes from 0 to 1.
The switching time SWT can easily be calculated numerically. Just generate L different sequences
 u
 u
 u
Environmental stochasticity
Coarse grained environmental stochasticity (sensu [37]) could affect the model system in several different ways; worker survival rate, worker productivity rate as well as season length can change from year to year. Here we restrict our analysis to the presumably most common effect in the context of bethedging; variation in season length [30,32,38,39]. Variation in season length might be caused by differences in both, the beginning and the end of the season. However, to model variation in season length it is sufficient to change the number of time steps L available within a single year, as allocation strategies always refer to the time passed since colony foundation.
To analyse the optimal investment strategy in variable environments a few essential modifications of the model system have to be made. As season length L_{j }varies between years j, reproductive output S_{Lj }will also vary and different years will contribute differently to overall fitness. As all colonies of a population simultaneously suffer from identical (and unpredictable) environmental fluctuations, the appropriate measure of overall fitness (F) of the strategy is the geometric mean of single year reproductive output [1,2,37]. We use the frequency distribution f(L) to describe the distribution of season lengths (L). Thus, each single year reproductive output S_{Lj }must be weighted according to the frequency x_{j }= f(L_{j}) of the corresponding season length L_{j}. For the expected long term fitness of a strategy we get
Bethedging analyses have been based on various assumptions about the shape of the frequency distribution of environmental quality. However, season length L = 0 must always be excluded from the distribution of possible seasons (f(0) = 0) [30], otherwise vanishing fitness in a single year with length L = 0 would imply that mean fitness F equals 0, whatever the shape of the distribution is like. Thus, every time discrete model must assume a minimum season length of at least one time step. Apart from this, distributions representing environmental fluctuations in modelling approaches may vary from uniform distributions ([33,40], typically characterised by their lower und upper boundary) to normal distributions ([30], characterised by mean and variance). Thus, we investigated optimal allocation strategies for uniform and (approximately) normal distributions as representatives of two extreme frequency distributions, assuming that natural distributions fall somewhere in between.
As we use a timediscrete model system, season length L_{j }in year j may only take integer values, while the normal distribution applies to continuous values. If season length is normally distributed (as assumed by [30]) with mean season length μ and variance σ^{2}, then the exponent x_{j }can be calculated as follows:
If season length is distributed uniformly between B  μ and B + μ [33] with mean season length μ and width B, then x_{j }does not depend on j (for B  μ <j <B + μ) and can be calculated as follows:
To compare the effect of normally and uniformly distributed season length on the optimal strategy we characterised both distributions by their variance; for the uniform distribution with width B it always is
In the following, graded strategies will be characterised by the width w of the transition zone, i.e. the number of time steps for which the optimal u_{i }surpasses a value of 0.05 but remains below 0.95. This is an appropriate measure as u_{i }usually increases monotonically over time (exceptions to this rule will be discussed later).
For stochastic environments with variable season length (L) the control function u_{i }(i = 0, ..., L) maximising fitness (F) cannot be calculated in the same straight forward way as before. In general, a recurrence method is required (see section Methods). Numerical results were calculated with the computer algebra system Mathematica 4.0 [41] and figures were plotted with the programming language R [42].
Estimating model parameters
To quantify the optimal allocation strategies we must estimate the relevant parameters for both, colony dynamics and environmental conditions. Lasioglossum malachurum is the best studied social halictid in Europe, where it shows enormous clinal variation in its social behaviour [35]. In its northern range it produces only a single worker brood and then a brood of sexuals, whereas in southern Europe it produces as many as three worker broods and a final gyne brood. Field observations of the halictid bee L. malachurum around Wuerzburg provide the most comprehensive data on colony development. In Wuerzburg a typical (foraging) season lasts about 80 to 120 days [26]. However, the nest cycle is organised in a sequence of active and inactive periods [10], and only during approximately half of this time foraging and provisioning is observed [26]. Thus, we assume a mean season length of L = 50 foraging days. Mean worker lifetime during foraging is about three weeks and survival rate per day can be approximated by q = 0.95 [35]. Worker lifetime efficiency in L. malachurum is about 3.2 offspring per worker [24,43]. In combination with the survival rate this results in a worker productivity rate of 3.2/21 ≈ 0.15. These values predict an overall production of about 40 sexuals, a typical value for L. malachurum colonies at Wuerzburg [24,35,44].
Variation in season length can only be estimated on a rather coarse level. Data of the duration of the yearly number of foraging days in halictids are only available for a few years (Weissel, unpublished). Thus, we based our calculations on numerous correlates of season length of halictid colonies. The length of the activity period is influenced by several factors with temperature (in particular soil temperature that influences the rate of brood development [26]), presumably being the most important. Furthermore, the availability of flowers for harvesting nectar and pollen might affect the duration of the flight season [45]. As direct measures of these variables are not available, we calculated the coefficient of variation for six possible indicators of season length based on daily temperatures and phenological data on the annual vegetation cycle provided by the German weather service (Deutscher Wetterdienst) since 1947: available cumulative degree days [d°C] above the zero development temperature of 10.5°C of L. malachurum (Weissel, unpublished), the number of days with mean temperature above 10.5°C, the time span between first flowering of Anemona (Anemona nemorosa) and first fruits of Common oaks (Quercus robur), the time span between first flowering of Anemona and grape gathering (MuellerThurgau), the time span between first flowering of Coltsfoot (Tussilago farfara) and first fruits of Common oaks, and the time span between first flowering of Coltsfoot and grape gathering. Anemona and Coltsfoot flower around the time of nest founding in L. malachurum and oak fruiting as well as grape gathering occur around the end of the emergence period of L. malachurum. There might be no perfect correlation of raw temperature data and flowering dates of plants with the activity period of bees. Nevertheless, there is probably a sufficient statistical concordance to warrant the use of these indicators to estimate variation in season length.
All estimates of the variability of season length for L. malachurum yielded similar values (Fig. 1) lending some confidence in the validity of these estimates. Thermal energy available for brood development typically varies by about 5% from year to year (Fig. 1a and 1d). Variation with regard to the dates of flowering of certain plants is less than 10% for all data sets (Fig. 1b, c, e, and 1f). If we use the maximum coefficient of variation (10% for the time span between first flowering of Coltsfoot (Tussilago farfara) and first fruits of Common oaks) observed in these data sets we would expect a typical standard deviation in the length of the foraging season of about 5 days. This is consistent with our observations of the colony activities of L. malachurum in the years 2002, 2003 and 2004 (Weissel, unpublished).
Figure 1. Frequency distribution of different indicators of season length: the yearly temperature sum [d°C] above the zero development temperature of 10.5°C of L. malachurum (Weissel, unpublished): n = 59 years (a), the time span between first flowering of Anemona (Anemona nemorosa) and first fruits of Common oaks (Quercus robur) : n = 27 years (b), the time span between first flowering of Coltsfoot (Tussilago farfara) and first fruits of Common oaks: n = 27 years (c), the number of days with mean temperature above 10.5°C: n = 59 years (d), the time span between first flowering of Anemona and grape gathering (MuellerThurgau) : n = 26 years (e), and the time span between first flowering of Coltsfoot and grape gathering: n = 26 years (f).
Numerical results
To analyse the general behaviour of the model system we will first consider the uniform distribution of season lengths [33]. From this simple case we will proceed to the normal distribution [30] and subsequently discuss the different results.
The optimal response of the model system to (uniformly distributed) fluctuating season length consists of two consecutive phases: 1) for low to moderate fluctuations of season length (Fig. 2a and 2b) the typical bangbang strategy with an abrupt transition between worker and sexual production is optimal. With increasing environmental variance the temporal position of the optimal switching point between growth and reproduction decreases (Fig. 2d and 2e) but – in contrast to Oster & Wilson's prediction [11] – the typical bangbang strategy performs better than a gradual change from worker to sexual production. 2) When variance in season length exceeds a critical level (Fig. 2c) the bangbang strategy is no longer adequate and a graded resource allocation strategy with an intermediate period of simultaneous production of workers and sexuals (and a humped strategy transition) becomes optimal (Fig. 2f). Graded resource allocation is characterised by a distinct phase of simultaneous production of workers and sexuals. It is a bet hedging strategy to avoid complete colony failure (Fig. 2f). With increasing variance of season length colonies start to reproduce earlier in their life cycle. However, while this temporal shift of the onset of reproduction is continuous (Fig. 3a), the transition between a pure bangbang strategy and graded resource allocation is predicted to be rather abrupt (Fig. 3b). Figure 3b also provides field data on the transition zone observed in L. malachurum at Wuerzburg, which is obviously not in accordance with model predictions. This important aspect is presented in detail at the end of this section.
Figure 2. Uniformly distributed season length and corresponding strategy response. The upper row shows the frequency distribution of season length for three cases with increasing variance (a: width B = 5, b: B = 10, and c: B = 15, mean μ always = 50). The lower row shows the corresponding optimal strategy transitions: the fraction of sexuals produced by the colony as a function of time (d, e, and f, worker productivity rate c = 0.15, survival rate q = 0.95). A small arrow indicates the temporal position of the optimal switching point in the case of a deterministic environment (SWT_{det}). Vertical grey lines indicate the boundaries of the distribution of season length. Figures d and e demonstrate that the optimal response of the system to increasing variation in season length is initially realised by an earlier switching from worker to sexual production. Graded strategies only emerge if environmental variation reaches a certain level and earlier switching alone would not be sufficient to buffer environmental fluctuations (c and f).
Figure 3. Onset of sexual production (a) and width w of the transition zone (b) between complete worker and complete sexual production as a function of the width of a uniform distribution of season length (B). w is defined as the number of time steps between the time when the strategy variable u_{i }surpasses a value of 0.05 for the first time and the time when u_{i }has finally reached at least 0.95 (and stays above this value for all remaining time steps). As long as w = 0, the reproduction strategy is bangbang (as in the deterministic case), but the optimal switching time moves to earlier points in time when variance increases (a). Model parameters: worker productivity rate c = 0.15, survival rate q = 0.95, mean season length μ = 50. The single emphasised point in the right figure denotes the combination of strategy transition and estimation of environmental variance observed for L. malachurum at Wuerzburg. The bars indicate the 95% confidence interval for the estimation of the mean w (between colonies, n_{1 }= 5 colonies) and the estimation of B (between years, n_{2 }= 48).
Further, the transition between colony growth and reproduction is not characterised by a monotonous increase in the amount of resources allocated to reproduction as suggested by Oster & Wilson (see Fig. 2.16 in [11]). Instead, the onset of the transition phase is characterised by a short pulse of nearly exclusive sexual production followed by a phase of simultaneous production of workers and sexuals before the colony finally ends with the exclusive production of sexuals. With increasing variance in season length the transition zone between pure colony growth and pure reproduction gets broader and the hump of worker production at the beginning of this zone becomes more pronounced. Tests with more restricted strategy sets (not shown here, see discussion) showed that this humped transition zone in fact significantly increases fitness compared to a monotonous sigmoid transition.
The pattern described above clearly depends on the specific form of the distribution of season length. When the uniform distribution is replaced by a normal distribution, both phases of system response (shift of SWT and onset of graded control) emerge again, but the graded control strategy is achieved earlier for lower environmental variance than in the case of equally distributed season length (Fig. 4 and 5). This is not surprising, as the normal distribution is not bounded and even allows for season lengths of only one day. The normal distribution is characterised by very smooth slopes on both flanks of the distribution. The rather smooth increase of the probability density on the left flank is reflected in a smooth increase in sexual production. In contrast to the case of the uniform distribution the strategy transition is thus nearly monotonic (Fig. 4f, in contrast to Fig. 2f).
Figure 4. Normally distributed variation in season length and corresponding strategy response. The upper row shows the frequency distribution of season length for three cases with increasing variance (a: width standard deviation σ = 2.9, b: σ = 5.8, and c: σ = 8.7, mean μ always = 50). The lower row shows the corresponding optimal strategy transitions: the fraction of sexuals produced by the colony as a function of time (d, e, and f, worker productivity rate c = 0.15, survival rate q = 0.95). A small arrow indicates the temporal position of the optimal switching point in the case of a deterministic environment (SWT_{det}). The vertical grey lines indicate the lower boundary of the distribution of season length at L = 1. The optimal response of the system to increasing variation in season length is initially realised solely by switching earlier from worker to sexual production, and graded strategies are realised when environmental variation increases (e, f).
Figure 5. Onset of sexual production (a) and width w of the transition zone between complete worker and complete sexual production (b) as a function of the width of a normal distribution of season length (given as the standard deviation). w is defined as the number of time steps between the time when the strategy variable u_{i }surpasses a value of 0.05 for the first time and the time when u_{i }has finally reached at least 0.95 (and stays above this value for all remaining time steps). As long as w = 0 (see b), the reproduction strategy is bangbang (as in the deterministic case), but the optimal switching time moves to earlier points in time when variance increases (a). Model parameters: worker productivity rate c = 0.15, survival rate q = 0.95, mean season length = 50. The single emphasised point in the right figure denotes the combination of strategy transition and estimation of environmental variance observed for L. malachurum. The bars indicate the 95% confidence interval for the estimation of the mean w (between colonies, n_{1 }= 5 colonies) and the estimation of B (between years, n_{2 }= 48).
The field observations of colony dynamics of L. malachurum yielded a rather broad transition zone with a period of approximately 11 days with simultaneous production of workers and sexuals ([24] and unpublished). For realistic standard deviations of season length of about 5 days this transition zone is far too broad to be explained as a bethedging strategy. According to our model, bethedging would predict a pure bangbang strategy for such a standard deviation under a uniform distribution of season length (Fig. 3), and only a transition phase of less than 5 days under the assumption of a normal distribution in season length (Fig. 5).
Discussion
Our analysis of the optimal resource allocation pattern in eusocial insect colonies clearly demonstrates that moderate fluctuations of environmental conditions (length of foraging season) will not necessarily foster the evolution of bethedging allocation strategies. This deviation from the rather intuitive predictions of Oster and Wilson is readily explained by the inherent buffering capacity of the bangbang allocation strategy; finite worker productivity and mortality rates determine an extended reproductive phase at the end of the season, when sexuals are produced exclusively. Thus, even if season length would be rather short, colonies could nonetheless expect certain fitness (successful production of sexuals) as long as the season ends after the onset of the reproductive phase. Consequently, rather high fluctuations in environmental conditions are needed to promote the evolution of graded allocation strategies with the simultaneous production of workers and sexuals.
The results of our model are rather robust against variation in model parameters (worker mortality, worker survival and mean season length). Parameter modifications within a plausible range did not change results markedly. It seems plausible to assume that increasing mean season length might reduce the effect of environmental variance, as identical environmental variance decreases relatively when mean season length increases. This is not the case. Increasing mean season length will just prolong the period of complete worker production, but not influence strategy transition. Even more, very short mean season length could result in strategies which start with the production of sexuals right from the beginning [33,4648].
As long as a season ends after the onset of the reproductive phase the pure bangbang strategy is buffered against complete reproductive failure. The switch from the bangbang strategy to a graded strategy thus strongly depends on the length of the reproductive phase. Worker efficiency and survival are the main determinants of the switching time in the deterministic case without environmental fluctuations and the optimal duration of sexual production decreases with increasing worker survival (q) and increasing worker efficiency (c) [8]. Populations in ideal conditions with high worker survival and high worker efficiency will thus switch to graded allocation strategies for much smaller variability of environmental conditions than populations that live under harsh environmental conditions.
Yet, at least for Lasioglossum malachurum the broad transition zone between pure colony growth and reproduction cannot be explained as an adaptive response to fluctuating environmental conditions. Even for normally distributed season length the observed transition zone is more than twice as long as predicted based on realistic estimates for the coefficients of variation for environmental fluctuations. Model results for uniformly distributed season length indicate that with more realistic distribution functions and observed variability of season length graded control is rather improbable.
According to our model, a transition phase between worker and sexual production is not necessarily characterised by a smooth continuous (sigmoid) increase in the production of sexuals as has been predicted by Oster and Wilson [11]. The specific form of this transition strongly depends on the frequency distribution of season length. For uniformly distributed season length with a very steep left flank, the transition zone is characterised by a peak in the amount of resources invested into reproduction of sexuals at the beginning of the transition phase. On the other hand, for the normal distribution, with its very smooth left flank the transition is characterised by a monotonic increase in the amount of resources invested into reproduction.
We have chosen the rectangular and the normal distribution because they are both commonly used types of frequency distributions representing the opposite ends of a continuum of distributions with increasingly steep flanks [30,33]. However, the normal distribution is an unbounded distribution while the minimum as well as the maximum length of a season is obviously bounded. Thus, assuming a normal distribution of season length may easily produce artefacts in the context of bet hedging. This can be seen, when the variance of season length is greatly increased. As we have to limit season length to a minimum of one day an increase in the variance of the normal distribution necessarily leads to steeper left flanks of this distribution. When we do this, the peak in resource allocation observed in the case of the uniform distribution emerges again. An increase of the lower boundary of the season length (in Fig. 3 and Fig. 5 we assume a minimum seasons length of one day, L_{min }= 1) will also reestablish the humped shape for predicted resource allocation. A lower boundary of L_{min }= 27 (about half of the mean season length) which cuts only 1% of the normal distribution in Fig. 4c will result in a prominent peak in the strategy curve. Immediately after the onset of the reproductive phase nearly 30% of the resources are invested into the production of sexuals. This rather high investment decreases subsequently to values around 15% before it rises again to end with a pure sexual phase (Fig. 6). In general this hump becomes more pronounced when either the length of the season becomes more variable or when the left flank of the density distribution of the season length becomes steeper (Fig. 2).
Figure 6. Optimal investment strategy (a), when 1% of the frequency distribution of season length is cut at the left side of the distribution (b). There is a prominent peak in sexual production just before minimum season length. Model parameters: worker productivity rate c = 0.15, survival rate q = 0.95, mean season length μ = 50, standard deviation σ = 8.7.
We have shown that variation in season length is not likely to be the reason for graded control in halictids. We have given two arguments. In general, environmental variation can already be buffered by simple bangbang strategies. Parameter and effect estimation for the sample species L. malachurum in fact lead to the conclusion that the environmental variation this species is exposed to is too low to necessitate graded control as an evolutionary answer.
Instead, we believe that alternative mechanisms can be responsible for the evolution of graded control in this species. For the sake of simplicity (and to accord with the simple model by Macevicz and Oster) we have assumed constant productivity and mortality rates during the season. However, it is reasonable that worker (per capita) productivity declines, when colony size increases. That this in fact occurs has been shown by Michener for several halictid species [49]. Although different theoretical approaches tried to analyse the effect of decreasing worker productivity within the framework of optimal dynamic resource allocation, the most straight forward analysis has never been performed: replacing the linear dependence of resource allocation on worker number (see eqns. 1 and 2) by a logistic relationship. Surprisingly, Macevicz and Oster [8] used such a relationship only for the worker equation (see eqn. 1 here) to estimate model parameters from field data, but not for the dynamics of the sexuals, thus inevitably favouring the early production of sexuals. Their remark, that saturation of productivity can promote graded control only for very restrictive parameter combinations seems premature, and further theoretical effort on this topic should be promising. Beekman et al.[9] investigated the effect of limited egg laying rate in bumble bees. This is equivalent to saturation of colony productivity, but unfortunately they allowed only for dichotomous strategy switches in their model, i.e. did not foresee the evolution of graded control. In this case the evolution of early switching is always accompanied by a waste of time. As the egg laying rate of L. malachurum queens is limited, too [35], this may be a reason for the graded allocation strategy observed in this species. First model calculations (unpublished) support this hypothesis. Thus, a thorough analysis should incorporate a broader set of strategic options to predict the influence of rate limitations on resource allocation strategies. However, it has to be kept in mind that workers of L. malachurum are not sterile. Thus, the queen's limited egg laying rate does not necessarily imply a saturation of egg production, as workers might provide additional eggs [35].
The original model of Macevicz and Oster [8] as well as our approach consider the colony (homologous to a single individual) as the unit of selection. As long as we assume that only a single trait is variable (and independent from others) and that all individuals of the colony do not have any further behavioural options, this perspective is also valid at the individual and the genetic level. However, since Macevicz and Oster there has been a wealth of theoretical and empirical analyses on individual worker fitness and the balance between gene, individual and colony level selection (see [50] for review). Selfishness, e.g. a worker disappearing into hibernation to become a queen the following year or leaving the colony for independent nest founding, may also account for 'graded control', even though this may be suboptimal at the level of the colony. For most eusocial halictids, workers probably have a variety of reproductive options ( see [51] for cases of 'workersized' queens in L. malachurum that enter hibernation to found colonies the following year). The same may be true for social polistine wasps, where the distinction between gyne and worker toward the end of the colony cycle (and even earlier in the colony cycle) is difficult [52]. The timing of the production of males is another aspect that has benefited from a genecentred analysis [53].
Conclusion
Up to now bethedging as an evolutionary response to variation in season length has been the main argument to explain field observations of graded resource allocation in annual eusocial insects. However, our analysis shows that the effect of moderate environmental fluctuations does not select for deviation from the classical bangbang strategy and that the evolution of graded allocation strategies can be triggered only by extreme fluctuations. Thus, the widespread belief that graded control in social insects is most probably a type of spreading of risk was premature.
Both, additional behavioural mechanisms at the colony level and genecentred or individualcentred approaches can provide promising alternative explanations. Detailed quantitative field or laboratory observations on resource allocation in eusocial insects are required to analyse the relevance of alternative explanations, e.g. logistic colony growth or reproductive conflict between queen and workers, for the evolution of graded allocation strategies.
Methods
The numerical solution of optimal control problems with objective functions averaging over different realisations (here: different season length) of the dynamic system (here eqns. 1 and 2) in a nonlinear way (here: geometric mean) cannot be achieved with standard dynamic programming [54]. However, nesting the dynamic programming approach within a recurrence procedure is a suitable way to find the optimal control function, if the iteration converges.
We first focus on the longest season length L_{max }possible with respect to the distribution of season length. The optimal value u_{Lmax }can be chosen independently from state and control function at earlier time steps, just based on maximising S_{Lmax. }Working backwards (dynamic programming) in time requires redefinition of the objective function. To find u_{Lmax1 }we have to maximise S_{Lmax }· S_{Lmax1}. This expression can be expanded in terms of state and control function values in time step L_{max1}. However, in general (for this and earlier time steps) it is not possible to find the value of u_{i }that maximises this expression without knowing the numbers of queens and workers in the current time step. Furthermore these values cannot be determined without knowing the values of u_{i }in previous time steps. This circularity can be broken by an iterative procedure. We first assume a trial solution u_{i}* for the control function and calculate the numbers of queens and workers in different time steps. Then an approximate value for u_{i }(not optimal yet) can be determined assuming the numbers of queens and workers just calculated. The approximate value can be calculated by derivating the objective function with respect to u_{i }symbolically and finding the root numerically. This procedure is repeated for all time steps back to i = 0. This yields an approximate solution for u_{i }(i = 0, ..., L_{max}).
Now the numbers of queens and workers can be recalculated using the approximate values of u_{i }from above. Next the optimal values of u_{i }can be recalculated, too. This process is repeated until the values of u_{i }have converged to the required accuracy (changes in the Euclidian norm of (u_{1}, .., u_{Lmax}) less than 0.001).
There might also be an analytical solution of the optimal control problem based on the application of Pontryagin's Maximum Principle, but we did not follow this line of approach ([33] provides detailed instructions).
Authors' contributions
All authors participated in the design of the study. OM developed the mathematical model, implemented and carried out the numerical calculations and drafted the manuscript. NW provided and presented the field data. ES and HJP conceived of the study, analysed and interpreted the numerical results and helped to draft the manuscript. All authors read and approved the final manuscript.
Acknowledgements
This study was supported by the Deutsche Forschungsgemeinschaft (SFB 554, TP C6). We would like to thank Daniela Giesen for computer simulations leading to the idea of this manuscript and PD Dr. Thomas Hovestadt and two anonymous referees for helpful comments.
References

Roff DA: The evolution of life histories. New York: Chapman and Hall; 1992.

Stearns S: The evolution of life histories. Oxford: Oxford University Press; 1992.

Cohen D: Maximizing final yield when growth is limited by time or by limiting resources.
Journal of Theoretical Biology 1971, 33(2):299307. PubMed Abstract  Publisher Full Text

Perrin N, Sibly RM: Dynamic models of energy allocation and investment.
Annual Review of Ecology and Systematics 1993, 24:379410. Publisher Full Text

Cole LC: The population consequences of life history phenomena.
Quarterly Review of Biology 1954, 29:103137. PubMed Abstract  Publisher Full Text

Kozlowski J, Teriokhin AT: Allocation of energy between growth and reproduction: The Pontryagin Maximum Principle solution for the case of age and season dependent mortality.

Macevicz S, Oster GF: Modeling social insect populations II: Optimal reproductive strategies in annual eusocial insect colonies.
Behavioral Ecology and Sociobiology 1976, 1:265282. Publisher Full Text

Beekman M, Lingeman R, Kleijne FM, Sabelis MW: Optimal timing of the production of sexuals in bumblebee colonies.
Entomologia Experimentalis et Applicata 1998, 88:147154. Publisher Full Text

Mitesser O, Weissel N, Strohm E, Poethke HJ: The evolution of activity breaks in the nest cycle of annual eusocial bees: a model of delayed exponential growth.
BMC Evol Biol 2006, 6:45. PubMed Abstract  BioMed Central Full Text  PubMed Central Full Text

Oster GF, Wilson EO: Caste and ecology in the social insects. Princeton, N.J.: Princeton University Press; 1978.

Michener CD: The bees of the world. Baltimore and London: The Johns Hopkins University Press; 2000.

Greene A, Akre RD, Landolt P: The aerial yellowjacket, Dolichovespula arenaria (Fab.): nesting biology, reproductive production, and behavior (Hymenoptera: Vespidae).

Smith FK: A colony of yellow jackets, Vespula pennsylvanica.

Blackith RE, Stevenson JH: Autumnal populations of wasps nests.
Insectes Sociaux 1958, 5:347352. Publisher Full Text

Haggard CM, Gamboa GJ: Seasonal variation in body size and reproductive condition of a paper wasp, Polistes metricus (Hymenoptera: Vespidae).

Greene A: Production schedules of vespine wasps: an empirical test of the bangbang optimization model.
Journal of the Kansas Entomological Society 1984, 57:545568.

Kolmes SA: Have hymenopteran societies evolved to be ergonomically effcient?
Journal of the New York Entomological Society 1986, 94:447457.

Martin SJ: A simulation model for colony development of the hornet Vespa simillima (Hymenoptera, Vespidae).

Roseler P: Unterschiede in der Kastendetermination zwischen den Hummelarten Bombus hypnorum und Bombus terrestris.

Müller CB, SchmidHempel P: Variation in lifehistory pattern in relation to worker mortality in the bumblebee Bombus lucorum.
Functional Ecology 1992, 6:4856. Publisher Full Text

Yanega D: Social plasticity and earlydiapausing females in a primitively social bee.
Proceedings of the National Academy of Sciences of the United States of America 1988, 85:43744377. PubMed Abstract  Publisher Full Text  PubMed Central Full Text

Yanega D: Environmental inuences on male production and social structure in Halictus rubicundus (Hymenoptera: Halictidae).
Insectes Sociaux 1993, 40(2):169180. Publisher Full Text

Strohm E, BordonHauser A: Advantages and disadvantages of large colony size in a halictid bee: the queen's perspective.

Hirata M, Cronin AL, Kidokoro M, Azuma N: Spatiotemporal variation of colony structure and eusociality level of the Japanese sweat bee Lasioglossum (Evylaeus) duplex (Hymenoptera : Halictidae).
Ecological Research 2005, 20:7579. Publisher Full Text

Weissel N, Mitesser O, Poethke HJ, Strohm E: The influence of soil temperature on the nesting cycle of the halictid bee Lasioglossum malachurum.
Insectes sociaux 2006, 53(4):390398. Publisher Full Text

Menu F, Roebuck JP, Viala M: Bethedging diapause strategies in stochastic environments.
The American Naturalist 2000, 155(6):724734. PubMed Abstract  Publisher Full Text

Menu F, Desouhant E: Bethedging for variability in life cycle duration: bigger and lateremerging chestnut weevils have increased probability of a prolonged diapause.
Oecologia 2002, 132(2):167174. Publisher Full Text

Kozlowski J, Ziolko M: Gradual transition from vegetative to reproductive growth is optimal when the maximum rate of reproductive growth is limited.
Theoretical population biology 1988, 34(2):118129. Publisher Full Text

Bradford MJ, Roff DA: An empirical model of diapause strategies of the cricket Allonemobius Socius.

Kaitala V, Mappes TY, loenen H: Delayed female reproduction in equilibrium and chaotic populations.
Evolutionary Ecology 1997, 11:105126. Publisher Full Text

Hopper KR: Riskspreading and bethedging in insect population biology.
Annual Review of Entomology 1999, 44:535560. PubMed Abstract  Publisher Full Text

King D, Roughgarden J: Graded allocation between vegetative and reproductive growth for annual plants in growing seasons of random length.
Theoretical Population Biology 1982, 22:116. Publisher Full Text

McNamara JM: Timing of entry into diapause: Optimal allocation to growth and reproduction in a stochastic environment.
Journal of Theoretical Biology 1994, 168:201209. Publisher Full Text

Knerer G: The biology and social behaviour of Evylaeus malachurus (K.) (Hymenoptera; Halictidae) in different climatic regions of Europe.
Zoologische Jahrbücher: Abteilung für Systematik, Ökologie und Geographie der Tiere 1992, 119(2):261290.

Sakagami SF: Sozialpolymorphismus bei Insekten. Probleme der Kastenbildung im Tierreich. Stuttgart: Wiss. Verlagsgesellschaft 1974 chap. Sozialstruktur und Polymorphismus bei Furchen und Schmalbienen (Halictinae); 257293.

Yodiz P: Introduction to theoretical ecology. New York: Harper and Row; 1989.

Bradford MJ, Roff DA: Bet hedging and the diapause strategies of the cricket A llonemobius fasciatus.
Ecology 1993, 74(4):11291135. Publisher Full Text

Wong TG, Ackerly DD: Optimal reproductive allocation in annuals and an informational constraint on plasticity.
The New Phytologist 2005, 166:159171. PubMed Abstract  Publisher Full Text

Taylor BE, Gabriel W: Optimal adult growth of Daphnia in a seasonal environment.
Functional Ecology 1993, 7(5):513521. Publisher Full Text

Wolfram Research Inc: Mathematica, Vers. 4.0. 4.0th edition. Champaign, Illinois: Wolfram Research, Inc; 1999.

Ihaka R, Gentleman R: R: A language for data analysis and graphics.
Journal of Computational and Graphical Statistics 1996, 5(3):299314. Publisher Full Text

Mitesser O, Strohm E, Weissel N, Poethke HJ: Optimal resource allocation in primitively eusocial bees: a balance model based on investment limitation of the queen.

Legewie H: Zum Problem des tierischen Parasitismus, I. Teil: Die Lebensweise der Schmarotzerbiene S phecodes monilicornis K. (= subquadratus) (Hymenoptera: Apoidea).
Zeitschrift für Morphologie und Ökologie der Tiere 1925, 4(3):430464. Publisher Full Text

SchmidHempel P, Durrer S: Parasites, oral resources and reproduction in natural populations of bumblebees.
Oikos 1991, 62(3):342350. Publisher Full Text

Sakagami SF, Munakata M: Distribution and bionomics of a transpalearctic eusocial halictine bee Lasioglossum evylaeus calceatum in northern Japan with reference to its solitary life cycle at high altitude.
Journal of the Faculty of Science Hokkaido University Series VI Zoology 1972, 18(3):411439.

Eickwort GC, Eickwort JM, Gordon J, Eickwort MA: Solitary behavior in a high altitude population of the social sweat bee Halictus rubicundus (Hymenoptera: Halictidae).
Behavioral Ecology and Sociobiology 1996, 38(4):227233.
ISI Document Delivery No.: UJ104; ArticleEnglish
Publisher Full Text 
Packer L, Jessome V, Lockerbie C, Sampson B: The phenology and social biology of four sweat bees in a marginal environment Cape Breton Island Nova Scotia Canada.

Michener CD: R eproductive effciency in relation to colony size in hymenopterous societies.
Insectes Sociaux 1964, 11:317342. Publisher Full Text

Ratnieks FLW, Foster KR, Wenseleers T: Conflict resolution in insect societies.
Annual Review of Entomology 2006, 51:581608. PubMed Abstract  Publisher Full Text

Richards MH, French D, Paxton RJ: It's good to be queen: classically eusocial colony structure and low worker fitness in an obligately social sweat bee.
Molecular Ecology 2005, 14(13):41234133. PubMed Abstract  Publisher Full Text

Reeve HK: The social biology of wasps. Ithaca, New York: Dornell University Press 1991 chap. The social biology of Polistes;

Fletcher DJC, Ross KG: Regulation of reproduction in eusocial Hymenoptera.
Annual Review Of Entomology 1985, 30:319343. Publisher Full Text

Mangel M, Clark CW: Dynamic modeling in behavioral ecology. Princeton, New Jersey: Princeton University Press; 1989.