The Celtics announced on Thursday that Allen, who missed Wednesday's game against the Atlanta Hawks after his bothersome right ankle swelled up, will not travel with the team to Toronto.
Allen is listed as day-to-day and could conceivably return at some point during the team's three consecutive road games, but would have to take his own flight.
"I don't know what's going on with it, but I do know he's not going to play tonight," Rivers said Wednesday of Allen's ankle. "It just swells, I guess. I don't know."
When asked whether Allen's ankle would eventually need surgery, Rivers added, "Ray would probably tell you more than I can tell you. I haven't heard that. I'm just more concerned that this is the second or third time it's happened."
For more Celtics coverage, visit CelticsBlog.