Do I always have to keep exactly in time with the CuraSwing app?

No, but the movement experience with the CuraSwing app improves if you walk at roughly the same tempo as the music. The music feedback for the arm swing is also designed for this tempo. If you walk much slower or faster, the music will sound less full. In addition, the rhythm of the music has a positive effect on your movement when it is roughly linked to the tempo of the music.