How to optimize groups of params/sites alternatingly?

Yes it’s fine to conditionally detach params. I often do that in the context of alternating optimization.