Machar fled South Sudan after fighting in the capital Juba in July.
The deposed vice-president fled South Sudan earlier in August.
Rebel leader Riek Machar has still not returned to Juba after clashes in July.
At least 120 cases of sexual violence and rape have been reported in the last three weeks.
Taban Deng Gai says he will take up Machar's role until the rebel leader returns to Juba.