For some reason, even when i am not in groups, I randomly lose karma... i have lost 5 karma in the last week and for some reason it bothers me

karma is not permanent.  every 24 hrs, your karma will degrade by 1 pt back towards 0... or neutral.  this allows people that have received loads of bad karma to eventually recover.  it also encourages people with good karma to continue doing things to keep their karma up.

 

I lost 3 at one time and i hadn't even joined any groups that day

Edit: It was after I updated my version from 1.8.1 to 1.8.2

you lose 1 pt every 24 hrs.  if you don't run groups, click find mesh, or log in for 3 days, then oQueue will attempt to sync time by checking for karma updates every 3 days.  without any additional karma, your karma would degrade by 3 pts in 3 days