Autonomous robot is a kind of complex cyber-physical system controlled by software. To support robots to operate in open environment in an effective and cooperative way is a great challenge for the researches and practices of control software of autonomous robot (CSAR). Adopting organization theory, this paper presents a multi-agent software architecture MaRSA (multi-agent robotic software architecture) that takes structure-in-5 organization style for CSAR. The software components of plan, dispatch, and execute behaviors of robot are independently encapsulated and explicitly separated, which lays architecture foundation for the flexible cooperation and continuous interactions among these components. The paper further proposes an accompanying behavior mechanism to enrich the interactions of observation actions and task actions, defines three kinds of accompanying relationships on the causality, temporal, and on-demand viewpoints, as well as designs a two-step dynamic decision algorithm DAAB (decision algorithm of accompanying behaviors) for planning accompanying behaviors. Two experiments are conducted in simulation robot environment and the real robot environment respectively, and the results show that comparing with the reactive behavior planning algorithm and BDI-based probabilistic planning algorithm, the proposed algorithm DAAB can produce plans that operate in open environment with high efficiency and low efforts to accomplish tasks.