Did i damage my tattoo?

i rubbed my four day old tattoo and some ink came off (and maybe skin?). did i damage the tattoo? i'm afraid i did it too hard!!