Josef Hofbauer and Joseph So:
Multiple limit cycles for three-dimensional Lotka-Volterra
Appl. Math. Lett. 7 (1994), no. 6, 65-70.
MR item for 96g:34063
Three-dimensional competitive Lotka-Volterra systems are considered
here with an equilibrium point in the positive octant of the phase
space. It is shown that among these systems there are such that have
more than one limit cycle around the equilibrium point. Actually, a
system with this property is explicitly constructed. The system
presented is permanent, i.e. the boundary of R^3_+ is
repelling; the equilibrium has a pair of imaginary eigenvalues and it
is repelling on its centre manifold. This implies the existence of at
least one limit cycle around the equilibrium. Then, by varying the
parameters slightly, the equilibrium undergoes an inverted
subcritical Hopf bifurcation, i.e. an additional unstable limit cycle
arises. Several open problems are raised in the paper. The authors
conjecture that the maximum number of limit cycles in such systems is
Reviewed by Miklos Farkas.
Copyright American Mathematical Society 1996, 1997
Our Conjecture was too naive. Several counterexamples have been found:
Zhengyi Lu and Yong Luo:
Three Limit Cycles for a Three-Dimensional Lotka-Volterra Competitive System with a Heteroclinic Cycle,
Computers and Mathematics with Applications 46 (2003) 231-238.
Gyllenberg M, Yan P, Wang Y (2006): A 3D competitive Lotka-Volterra system with three limit cycles: a falsification of a conjecture by Hofbauer and So. Appl Math Lett 19: 1-7.
Xinze Lian, Zhengyi Lu,Yong Luo (2008):
Automatic search for multiple limit cycles in three-dimensional Lotka-Volterra competitive systems with classes 30 and 31 in Zeeman's classification.
Journal of Mathematical Analysis and Applications 348 (2008) 34-37.
Gyllenberg M, Yan P (2009): Four limit cycles for a three-dimensional competitive Lotka-Volterra system with a heteroclinic cycle. Comput Math Appl 58:649-669
Recently, there appeared a critial review, pointing to errors in some of the above papers:
Pei Yu, Maoan Han, Dongmei Xiao:
Four small limit cycles around a Hopf singular point in 3-dimensional competitive Lotka-Volterra systems.
Journal of Mathematical Analysis and Applications 436 (2016), 521-555.