How can I protect myself against my pervert landlord?

My landlord keeps finding reasons to stop by my home during the day while I am alone. Yesterday he came by and I didn't answer the door. He then came back 20 minutes later with a hammer and proceeded to pound on the door with it. He is a weird little guy and has been coming onto me for years. One time he tried exposing himself to me. I was able to make him stop that time, but what I am afraid of is that he seems to be getting more ually aggressive, He is always trying to grope at me. The scary thing is he has a key to the door. Our home is in need of repairs and he insists on doing them all himself, giving him the opportunity to oogle me daily. Are there laws against landlords who ually har their tenants. (It is bordering on stalking) What can I do?? Is it legal for me to change the locks since he has a key??