I understand that wall units are not meant to get removed from the wall, since otherwise players would always have a full offense.
I suggest a tiny change though: the ability to exchange troops of the same rarity.
E.g. exchange 24 featherbows for 6 templars.
One reason is tactics. Assuming one mission requires 12 templars and you have 24 featherbows and 24 legionaires in your army and 10+ slots of templars already on your wall.
1) You will need to put the units you normally play on the wall
2) You will need to build even more units you don't need (imagine the wall is full)
3) Finish the mission
4) Put the new units on the wall as well
5) rebuild the units you had before
This is kind of a frustrating process.
Another usecase are sieges: you want golden catapults to destroy some walls, then you want to do field battles again.
With the current slot caps you have to rebuild all golden units each time, so your gameplay gets static and unflexible.