Today's Internet video streaming systems employ buffering and multiple retransmissions to guarantee the correct reception of each packet, which leads to high latency in media delivery. In this work, a novel scheme of dynamic management of the dependency across packets is proposed using optimal reference picture selection at the frame level that adapts to the channel. The optimal selection of the reference picture is achieved with a rate-distortion framework, which minimizes the expected end-to-end distortion given a rate constraint. The expected distortion is calculated based on an accurate binary tree modeling with the effects of channel loss and error concealment taken into account. Experiments demonstrate that the proposed scheme provides significant performance gains over a simple INTRA-insertion scheme. The increased error-resilience eliminates the need of retransmission, which makes it possible to reduce the latency from 10-15 seconds to a few hundred milliseconds, with similar video quality maintained.