Should i let it go or not?

I was with him for almost two years and we had a very bad break up, two months ago. We didnt have any contact until he sent me a text message saying just a plain "hi" a month after we broke up. I ignored it and a week later he sends one more text message asking me if i cheated on him during the time we were together. I got really angry and i replied asking him if he was serious. He never replied back and now, two months after our break up he sends me another message saying that we really have to talk because we need to clear up something we left hanging. Of course he means that he wants to know if i ever cheated on him! I never did, I loved this man deeply but even the time we were a couple he used to not trust me completely and accuse me off cheating on him or even accuse me of THINKING about cheating on him! I replied asking what was that issue we left hanging and he didnt reply. Three days have gone by and i text him again saying that if he is not going to tell me whats up then he should not text me randomly again. Once more no reply. I tried a lot in this relationship, he did too but this persistence of him was unbearable. Although i stayed with him because i couldnt imagine my life without him; he was a part of me. Until he finally broke up with me. We ended it really bad because i was asking him to work it out and he did not want so we left it there and we have never spoken again, until he started texting me like i mentioned above. Now I'm VERY confused, I was getting over him, i was slowly moving on and i was okay, utnil he appeared again back in my life. I still have feelings for him I dont know how to handle it, and now that i was harsh in the last text message i sent him i am sure he is accusing me of being selfish and ungrateful. i am not! i appreciate him and our time together he was a good guy but i wont let my self esteem reach the bottom. im not a cheater! I really want to clear this misunderstanding up without seeming needy. Should i just let it go o