The inference techniques for probabilistic knowledge bases have recently attracted significant attentions. In most off-the-shelf existing systems, the inference is mainly implemented based on batch processing and thus not suited for online querying. This paper proposes an online inference approach based on approximate factors for probabilistic knowledge bases, so as to provide a way to reuse those inferred results to calculate the marginal probability for the query variable. In this approach, a subgraph is extracted first, taking the query variable as center; then some approximate factors are attached to simulate the influences from the variables outside the subgraph; and finally, the marginal probability of the query variable is calculated by the clique tree algorithm. Experiments show that compared with existing algorithms, the presented approach can achieve a better tradeoff between accuracy and time.