This is intended behaviour. When you zone into Timewalking Dungeons, certain buffs will be cleared and will need to be re-cast. This is necessary for clean down-scaling.
In the case of Arena, this is to stop players loading up on various buffs that they would not normally be able to receive while in an Arena match.