Available bit rate (ABR) flow control is an effective measure in ATM network congestion control and traffic management. In high-speed ATM networks, the switch performance lies on the algorithm simplicity in some degree. Although the simplicity of binary flow control is very attractive, the queue length and allowed cell rate (ACR) controlled by the standard EFCI algorithm oscillate with great amplitude, which must have negative impact on the performance, so its applicability is doubted, and then the explicit rate feedback mechanism, which is relatively complex but effective, is introduced and explored. In this study, based on the ABR flow control model, a novel binary ABR flow control algorithm is put forward using the approach for designing the sliding mode variable structure controller in robust control theory, jointly applying the congestion detection mechanism based on the probability. The new algorithm avoids the self-oscillation induced by the nonlinear component in the standard EFCI algorithm, which is very favorable for utilizing the simplicity of binary flow control mechanism to optimize the switch performance. The simulation results show that the sliding mode variable structure controller greatly constrains the oscillations of ACR and queue, smoothes the delay jitter, which provides the reliable implementation mechanism for QoS guarantee in ATM network.